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Description 

FIELD OF THE INVENTION 

s The present invention relates generally to diagnostic testing, and more particularly to methods and apparatus for 

diagnostic testing using evidential reasoning and for determining when sufficient infornnation has been collected to 
make a valid diagnosis. 

BACKGROUND OF THE INVENTION 

10 

The assignee of the present invention has previously developed a portable interactive diagnostic tester for fault 
isolation in complex electromechanical systems called POINTER. POINTER utilizes a model-based, matrix-implement- 
ed inferential reasoning method of diagnostic analysis. The diagnostic model is created using the STAMP (System 
Testability and Maintenance Program) program also available from the assignee of the present Invention, ARINC Re- 
ts search Corporation. STAMP is an interactive modeling tool designed to permit analysis of the testability of a system 
and generation of static fault isolation strategies. STAMP assumes that the system design is known, tests can be 
specified that measure the goodness and badness of system elements, and sequences of test outcomes can be used 
to localize or isolate failed system elements. The STAMP diagnostic model describes the flow of information through 
a system to be diagnosed. The information flow is represented as a set of diagnostic tests (informatk>n sources), a set 

20 of diagnoses (conclusions to be drawn), and a set of logical relationships between the tests and the conclusk>ns. In 
addition, the model includes parameters for weighting a test selection optimization process, grouping tests and con- 
clusions, and specifying types of tests to be performed (e.g. functbnal tests, conditional tests, or asymmetric tests). 
Under the STAMP methodology, an assumption is made that only one fault exists or that specific multiple faults can 
be specified and treated as a single fault, and thus that only one conclusion is true for any one diagnostk: analysis. (If 

25 the system under test actually has multiple faults, they can usually be successively isolated by repeating the diagnostic 
analysis, unless a "hidden" failure causes the failure which is isolated or a multiple failure looks like an independent 
single failure.) Evidence from an individual test flows to a set or sets of diagnostic conclusions as either support or 
denial evidence in accordance with the defined logical relationships. That is, a logical proposition consisting of a set 
of dependency, or support, functions is established or defined which identifies the conclusions related to a given test 

30 and allocates the evidence generated by a test measurement to the set of related conclusions. An example of a higher- 
order proposition relating a test \^ to a set of conclusk>ns c^, C2 c^ is: 

If (ts) then c^ or C2 or ....c^ implication 

If {"c^ and ""02 and ^ . .Cn) then "X^ negation 

If (""ts) then '"Ci and '"C2 and . ..'"Cn symmetry 

35 Each unique test has two different support functions, one for each type of evidence (support or denial). There is 

thus a set of conclusions that are supported and a set of conclusions that are denied by a given test outcome. That Is. 
every conclusion in the model is supported or denied by each test outcome. A test supports a conclusion when the 
test outcome is consistent with a conclusion value of true. A test denies a conclusion when its outcome is not consistent 
with a conclusion value of true. In addition, component-to-component dependencies in the system are not modelled, 

^0 and thus an assumption is made that the evidence for a given conclusion is determined solely by the tests that are 
related to it, not by other concluskDns that may resemble its component parts. 

The test measurement results are evaluated and specified as indicating that the tested system element is "Good, 
" 'Bad" or "Unknown" (results not available). A Bad test outcome is equated to be a logically True value of the proposition 
that corresponds to the test; and a Good test outcome equates to a logically False value of the corresponding test 

45 proposition. In the example given above, testing would evaluate the outcome of tg. and then the truth or falsity of other 
propositions and conclusions are inferred using the applicable set of support functions. Thus, the STAMP process can 
be generally described as solving a set membership problem. If a given test is Bad, it implies one of n higher-order 
causes. If another test is Good, it Implies that all of the potential causes of its being Bad are not possible conclusk>ns. 
so that the complementary set (all potential causes that are still possible) is possible. STAMP operates to ascertain 

so the intersections of the possible conclusion sets. 

The STAMP paradigm has proven successful in solving a large number of testability and fault-isolation problems. 
However, many important problem domains Involve uncertainty in both the statements of relation between tests and 
conclusions and the interpretation of the test measurements. For example, the model assumes that the relationships 
betwe n the information sources and the diagnostic conclusions are clearly discernible, well defined, and unambiguous. 

ss It may not be possible, though, to establish a mod I which is both accurate and complet , i. ..arnod 1 in which ach 
conclusion is properly related to each test result, and all of the conclusions which should be relet d to a test result are 
specified. Further, the test information may be ambiguous, difficult to interpret, or the user doing the testing and inter- 
preting may have inadequate skill levels to produce reliable test results. Thus, an indicator is needed which measures 
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how well the modelled r lations are fitting the problem to which they are being applied, A number f different approaches 
have been developed In the artificial intelligence field to deal with the uncertainty created by these problems. Generally 
ref rred to as 'reasoning under uncertainty," exampi s of such approach s include the use of certainty factors, pr b- 
abilistic reasoning and weight d causal networks, in addition, there are various logics that tak into account aspects 

5 of uncertainty such as predicate calculus, multi-valued logics, modal logics, non-monotonic reasoning, intuitional logic, 
truth maintenance, "fuzzy" logic, and Dempster-Shafer evidential reasoning. 

However, it is difficult in general to integrate reasoning under uncertainty concepts with the massive structure and 
processes associated with a complex, matriximplemented inferential reasoning process like STAMP. Further, although 
the Dempster-Shafer evidential reasoning approach is well adapted for use in an inferential diagnostic process like 

10 STAMP, even this approach to dealing with uncertainty creates a number of serious problems. The Dempster>Shafer 
technique is flawed in the way it handles total uncertainty. II any test is performed that provides any evidence in support 
of some conclusion, then uncertainty is reduced-even in the event of a conflict with known information. Ultimately, as 
further testing is conducted, uncertainty disappears altogether, independent of the test results. Also, applying uncer- 
tainty calculations to an inferential system makes the criteria for determining when a diagnostic conclusion may be 

IS properly drawn unclear There is thus a need to determine at what point enough information has been gathered to 
declare a valid diagnosis, or, in the context of fault isolation, to determine the point at which additional testing is providing 
minimal useful new information. 

Examples of commonly used diagnostic approaches include protocol or decision tree based systems and simula- 
tion models; and so called "intelligent" systems such as rule-based expert systems as discussed below. 

20 

Shortliffe, E.H. Computer Based Medlclal Consultations: MYCIN, American Elsevier, New York, 1976. 

The first "successful" artificial intelligence application was the MYCIN system developed by Edward Shortliffe at 
Stanford University. This book was based on Shortliffe's Ph.D. Thesis and describes the use of heuristic rules, backward 
2S chaining, and an uncertainty mechanism called "certainty factors" as applied to the problem of diagnosing infectbus 
diseases. The MYCIN system was specifically designed to serve as a decision aid for physicians and physician as- 
sistants in the diagnostic process. This system was completely passive and gave no indications of whether more testing 
was required to make a diagnosis. Rather, only confidence levels were provided with recommended diagnoses and 
treatments. 

30 

Davis, R., "Diagnostic Reasoning Based on Structure." Artificial intelligence. Vol. 32. pp. 97-130. 1987. 

In an attempt to move from the medical domain to the electronics domain, Randall Davis developed a model-based 
approach to diagnosis focusing on structural and behavioral characteristics of the system being tested. The models 
3S used structure to define signal flow through the circuit and behaviour to reason about responses to tests. Davis' model- 
based approach was also passive and included no mechanism for reasoning under uncertainty Later work on the 
model applied entropy-based search to optimize testing and moved the approach from passive to active, but to our 
knowledge, uncertainty was still left out. Consequently, termination was deterministic, meaning it was based on applying 
test results to sets of hypotheses until the smallest set that could explain the test results was obtained. 

40 

Pipltone, F., "The FIS Electronics Troubleshooting system," lEE Computer, pp. 68-75, 1986. 

While working at the Naval Research Laboratory (NRL), Frank Pipitone developed a model-based diagnostic sys- 
tem (the Fault Isolation System -FIS) which was based strictly on structural characteristics of electronic systems. These 
<s models are called "dependency models" in the diagnostics literature. The approach was heavily influenced by Davis, 
but did not include the behavioral knowledge. Further, Pipitone added probabitistk: reasoning to the inference process. 
FIS was an active diagnostic system (i.e. test choice was optimized), but termination was based on testing with certain 
outcomes as in Davis' approach. In other words, the uncertainty of the answer had no bearing on whether or not FIS 
would provide an answer, thus degrading the accuracy in the answer 

so 

Allred, Lloyd, "Neural Networks In Automatic Testing of Diode Protection circuits," AUTOTESTCON '89, pp. 
181-186. 

Neural networks are being applied in increasing frequency to the problem of electronic system diagnosis. This 
55 paper by Lloyd Allred provides an xample of one application of neural networks for circuit diagnosis as applied to 
Zener diodes. The d scrib d approach involves extracting rel vant f atures from th performanc of the diode and 
using these features to train a neural network. The neural network was based on the architecture of a self-organizing 
feature map in which feature values are grouped into cells based on distance. The cells were then labelled with fault 
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identifiers so that when a c II fired, the corresponding fault was identified. Cell sizes were adjusted through training so 
that each cell only identified a single fault. The approach described yields a network that is highly specific, system 
specific, and test specific. To be us d for testing other units, a new network would need to b constructed. This, in fact, 
is typical of neural network applicatbns for diagnosis. 
s Exannples of known methods which can be used to determine levels of certainty (or "pseudo" probabilities) have 

b n cited hereinabove, and include the following: 

McLeish, M., "A Note on Probabilistic Logic." AAAI>e8, pp. 215-219, 1988 

10 The question of how one applies probabilistic logic to reasoning systems is addressed in this paper. The question 

addressed is not specific to diagnostics but concerns whether two forms of probabilistic reasoning (namely maximum 
entropy and projection) yield the same result. The benefit of the article is on analysis of reasoning performance to 
determine validity of the underlying mathematics. It addresses neither efficiency of reasoning nor termination criteria. 

IS Zadeh, L., "Fuzzy logic." IEEE Computer, pp. 83-93, April 1988 

Lotfi Zadeh defined a new method of reasoning under uncertainty that he called 'Fuzzy Logic' Fuzzy logic, as 
defined In this paper, is based on the notion of a fuzzy set. Rather than declaring an element to be a member of some 
set, the mathematics allows a "lever of membership to be associated with the element. Fuzzy logic then consists of 
20 methods for combining fuzzy membership values to determine, given multiple pieces of evidence, whether or not some 
set is characterized by the data. 

Shafer, G., A IWathematical Theory of Evidence, Princeton University Press, pp. 35-73, 1976 

2S Believing Bayesian probability theory was powerful but too unwieldy to apply In practice. Shafer devised a modi- 

fication to Bayesian inference in which he analyzed available evidence to determine levels of belief and denial in a 
hypothesis. Shafer's theory of evidence provided a means for determining belief intervals for the hypotheses rather 
than specifying a single probability value. Shafer believed single probability values were too difficult to interpret because 
they did not provide sufficient Information about the reasoning process. This work was not explicitly applied to diagnosis; 

30 however, its widest application was in traditional classification tasks (of which diagnosis Is an example). The mathe- 
matics dkJ not Include methods for optimization or termination but considered all available evidence. 

D mpster, A. P., "A Generalization of Bayesian Inference," Journal of the Royal Statistical Society. Series 8, 
pp. 205-247, 1968 

35 

Dempster extended the work by Shafer to provide a method for combining evidence from multiple sources. In 
particular, Dempster defined a 'rule of combinations" that permitted an iterative analysis of evidence. The resulting 
reasoning mechanism is now referred to as "Dempster-Shafer theory" and forms the basis of many reasoning systems 
most notably sensor fusion systems. Again, Dempster assumed the availability of the needed evidence and did not 
40 focus on optimization for evidence selection or termination of the reasoning process. 

The rule-based expert system is perhaps the most common of the 'intelligent' diagnostic systems. Experts are 
inten/iewed to determine how they diagnose, and rules are written to describe the process. Uncertainty is nriost fre- 
quently handled using probabilistic inference or certainty factors. 

The objects of the invention are achieved by the apparatus as set forth in claim 1. The dependent claims are 
45 directed to preferred embodiments of the invention. 

In accordance with the present invention there is provided automatic test data evaluation apparatus for use in a 
diagnostic tester that declares a diagnosis on the basis of localisation or isolation of failed system elements of a system 
having a defined number of functional elements, said apparatus comprising: 

50 means for selecting the next test to be performed on the system to be tested; 

means for generating the corresponding test data and supplying them to the system to be tested; 
means for sensing and collecting predetermined system operating parameters; 

evaluating means, responsive to at least one inputted test signal corresponding to test data Sj relating to at least 
one predetermined parameter of the system, for producing using a probabilistic evidential reasoning method: 
ss first and second candidate signals corresponding r sp ctively to first and second possible diagnos s of th con- 

dition of the system r spectively having the first and s cond high st levels of certainty of being valid, and 
first and second c rtainty signals corresponding respectively to values of said first and second highest levels of 
certainty; and 
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characterized by testing suffici ncy means, responsive to said ftrst and second certainty signals, for producing 
an output signal indicative of whether sufficient test data has been evaluated to declare a diagnosis, said testing suf- 
ficiency means comprising a neural network having an output node and at least thr e input nodes respectiv ty rec iving 
said first and second certainty signals and a pred t rmined bias signal as an input, said input nodes having weighted 
s outputs, the output weights corresponding to a model of expert data correlating combinations of certainty signals with 
at least one expert opinion regarding the ability to declar a diagnosis on the basis of ach combination of certainty 
signal values. 

The present invention provides improved diagnostic testing methods and apparatus which deal with uncertainty 
and overcome the deficiencies of conventional approaches. 
10 The present invention also provides improved evidential reasoning based diagnostic testing methods and appa- 

ratus which overcome the limitations of the Dempster-Shafer approach. 

The present invention also provides methods and apparatus for determining the sufficiency of testing in diagnostic 
testing methods and apparatus which deal with uncertainty. 

The present invention also provides a neural network method and apparatus for determining the sufficiency of 
IS testing in diagnostic testing methods and apparatus dealing with uncertainty wherein the neural network can be trained 
to haye generic application to different diagnostic models and different systems. 

Typically said evaluating means also produces an uncertainty signal corresponding to a measure of the uncertainty 
that the evaluated at least one test signal can be validly evaluated; and 
said neural network comprises: 

20 

a set of first, second, third and fourth input nodes each having first, second and third outputs weighted by a pre* 
determined fixed weight, said first and second input nodes respectively receiving. said first and second certainty 
signals as an input, said third input node receiving said uncertainty signal, and said fourth input node receiving 
said predetermined bias signal; 

2S a set of first, second, third and fourth intermediate nodes each having an output weighted by a predetermined fixed 

weight, and said first intermediate node receiving the sum of the first weighted outputs of said input nodes as an 
Input, said second Intermediate node receiving the same of the second weighted outputs of said input nodes as 
an input, said third intermediate node receiving the same of the third weighted outputs of said input nodes as an 
Input, and said fourth intermediate node receiving said predetermined bias signal; and 

30 an output node, responsive to the sum of said weighted outputs of said intermediate nodes, for producing said 

output signal, the output weights of said input and intermediate nodes corresponding to a model of expert data 
correlating combinations of certainty and uncertainty signal values with at least one expert opinion regarding the 
ability to declare a diagnosis on the basis of each combination of certainty and uncertainty signal values, said 
testing sufficiency means thereby also evaluating sakJ uncertainty signal to produce said output signal. 

35 

Typically said evaluating means also produces a signal S|sj corresponding to a measure of the level of certainty 
that a diagnosis not corresponding to sakJ first or second candidate signals or to said uncertainty signal Is a valkd 

diagnosis; and 

said testing sufficiency means also evaluates said signal S^g to produce said output signal. 

40 The diagnostic tester typically further comprises means for storing a model of predetermined intormatbn flow 

relationships within the system under test; means for associating the test signals with sets of possible causes in ac- 
cordance with the model, and for ascertaining the intersections of the sets of possible causes for combinations of test 
signals, the candidate signals being derived from the intersections; and means for determining for each candidate 
signal a measure of the level of certainty that it represents a valid diagnosis. 

4S Typically the diagnostic tester further comprises means for defining a plurality of conclusions corresponding to 

possible diagnoses of the system; means for assigning a confidence value to each successively inputted test signal; 
means for calculating the support and denial for each conclusion based on the confidence value of the currently inputted 
test signal; means for determining if the currently inputted test signal conflicts with a then selected set of said conclu- 
sions corresponding to candidate signals; means for obtaining a measure of internal conflict based on accumulated 

so total support values and the current test signal support values for all of the conclusions; means for computing the 
present value of the uncertainty signal based on the number of test signals evaluated, if the present test signal is not 
a conflict; and based on the number of test signals evaluated and the number of conflicts. If the present test signal is 
a conflict; means for computing a new total support value for each conclusion based on the old support value, the 
present test signal support valu and confidence value, an old uncertainty value, and the measure of internal conflict; 

ss means for computing a new uncertainty value bas d on the old uncertainty valu , th pr s nt test signal confidence 
valu and the measure of Internal conflict; means for normalizing the total support value for each conclusion based on 
the old total support value, and the present uncertainty signal value; means for computing a new total denial for each 
conclusbn based on an old total denial and the present test signal denial value; means for computing th present 
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plausibility for each conclusi n based on the present total denial and the number of test signals evaluated; means for 
selecting the current set of candidate signals based on the present total support and the plausibility tor each conclusion. 

While a purely inferential form of STAMP provides a very good method for optimizing the choic f the next test 
to maximize the efficiency of the diagnostic process, this m thod breaks down when an videntlal reasoning approach 
5 is adopted. Heretofore, the problem of test selection has not been addressed in the field of evidence combination. 
Second, the process of identifying the conclusions, obs n^ations and relationships that should be involved In a given 
evidential diagnostic process Is much more difficult than with a strictly inferential process. 

Typically the diagnostic tester further comprises an arrangement for testing a system according to a set of defined 
relationships between a plurality of sources for providing Information about the system and a plurality of conclusions 
10 about the nature or condition of the system, said tester comprising first means for successively selecting an Infornnatlon 
source from among the plurality of information sources for interrogation according to the relative amount of information 
provided by each Information source until all of the Information sources have been selected or the Information which 
would be provided thereby has been determined; 

second means for successively further selecting an information source from among the plurality of information 
75 sources for Interrogation according to which Informatbn source will Increase the level of certainty that a selected can- 
didate conclusion constitutes a valid diagnosis of the system being tested. 

These and other features and advantages of the present Invention are disclosed In or apparent from the following 
detailed description of preferred embodiments. 

The preferred embodiments are described with reference to the accompanying drawings, wherein like features 
20 have been designated with like reference numbers, and wherein: 

Fig. 1 is a diagrammatic view of a first embodiment of a diagnostic tester according to the present Invention. 
Fig. 2 is a schematic block diagram of the basic functional components of the diagnostic tester of Fig. 1 . 
Fig. 3 is a schematic block diagram of the control and diagnostic analysis CPU of Fig. 2. 
Fig. 4 is a diagrammatic view of a second embodiment of diagnostic tester according to the present invention. 
2s Fig. 5 is a schematic block diagram of an automatic system tester Incorporating diagnostic testing apparatus in 

accordance with the present invention. 

Fig. 6 is a schematic block diagram of a system controller incorporating diagnostic testing apparatus In accordance 
with the present invention. 

Fig. 7 is a schematic block diagram of the controller and diagnostic test signal evaluating sections of a diagnostic 
30 tester according to the present invention. 

Fig. 6 is a general flow chart of a diagnostic process performed by a diagnostic tester in accordance with the 
present Invention. 

Fig. 9 is general flow chart of a preferred diagnostic process performed by a diagnostic tester in accordance with 
the present Invention. 

3S Fig. 10 is a more detailed flow chart of a preferred test selection process performed by a diagnostic tester in 

accordance with the present invention. 

Fig. 11 is a more detailed flow chart of a preferred evidential reasoning based process performed by a diagnostic 
tester in accordance with the present Invention for selecting a conclusion from among a plurality of conclusions as a 
diagnosis based on Inputted test signals. 
^0 Fig. 12 is a diagrammatic illustration of a first embodiment of a neural network in accordance with the present 

inventbn for determining the sufficiency of the testing by a diagnostic tester 

Fig. 1 3 Is a diagrammatic illustration of a second embodiment of a neural network in accordance with the present 
invention for determining the sufficiency of the testing by a diagnostic tester. 

4S DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Referring to Figs 1-6. the diagnostic testing methods and apparatus of the present invention advantageously are 
incorporated in diagnostic testers which are adapted to receive test data input from an operator or from other sources 
(Figs. 1-4); in automatic system testers which are adapted to automatically both test a system and analyze the test 

50 results (Fig. 5); or In automatic system controllers which sense the condition of the controlled system and control the 
operation of the system in response thereto (Fig. 6). The present invention may also readily be embedded within 
systems having self-diagnosis capabilities as part of the self diagnosis architecture. As shown in Figs. 1-3, one form 
of manual input diagnostic tester which is portable has a separate housing 11 which contains the tester components; 
a display 12 mounted in housing 11 for displaying test results, diagnoses, user prompts and similar types of information; 

ss a k yboard area 13 comprising a plurality of data ntry k ys 14 and a plurality of command keys 15 for entering test 
data and user commands to control the operation of the test r; conventional signal g nerating circuitry 16 responslv 
to actuation of k ys 14 and 15 for generating digital electrical test and command signals, respectively, corresponding 
to the test data and commands entered by the operator; and a control and diagnostic analysis central processing unit 
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(CPU) 17 for c ntrolling th operation of the t ster and performing the diagnostic analysis routines stored therein. As 
is conventional, CPU 17 comprises a processor I7a for performing control and analysis operations under control of 
an operating program stored in a read-only memory (ROM) 17b, and a furth r data memory 17c for storing inputted 
test and command signal data and the data general d In the course of operation of processor 17a. 

s Referring to Fig. 4, another form of a diagnostic tester in accordance with the present inventbn comprises a con- 

ventional general purpose or dedicated computer system 20 comprising a display 22, a CPU 24 programmed to perform 
diagnostic analysis routines, and an input interface 26 for receiving test data and command signal input. As will be 
appreciated by those skilled in the art, CPU 24 has the same general functional organization as CPU 17 shown in Fig. 
3. Input interface 26 advantageously comprises a conventional keyboard or other user interface device for generating 

10 test data and command signals in response to operator actuation of the device, or a conventional peripheral interface 
device for receiving test data and command signals from a peripheral source such as a storage memory or system 
controller; and converting, if appropriate, the test data and command signals to digital signals in a form compatible for 
processing by CPU 24. 

Referring to Fig. 5, an automatic system tester 30 for testing a system A whbh incorporates a diagnostic tester in 

IS accordance with the present invention comprises a controller and diagnostic test signal analyzer 32 for generating 
control signals to control the operation of tester 30 and for analyzing test signals indicative of the nature or condition 
of system A which are generated by tester 30; a test generator system 34 for performing predetermined tests on system 
A responsive to control signals generated by controller/analyzer 32 and for generating the test signals corresponding 
to the test results; and a display 36 responsive to control signals from controller/analyzer 32 for displaying test infor- 

20 mation, diagnoses, user prompts and the like. 

Referring to Fig. 6, a diagnostic tester in accordance with the present invention is shown Incorporated in a system 
controller 40 controlling the operation of a system A. System controller 40 comprises a sensor/control unit 42 for sensing 
predetermined system operating parameters and for generating control signals for controlling the operation of system 
A; and a diagnostic tester 44 for evaluating the sensor data and generating output signals used by sensor/control unit 

2S 42 for generating the control signals. 

The diagnostic tester of the present invention has broad application in determining the nature or condition of a 
variety of systems, ranging, for example, from determining faults and malfunctions in electronic or electromechanical 
systems or manufacturing processes, to selecting the appropriate control response according to the sensed state of 
a controlled system, to identifying a disease or other abnormal condition in an organism, to Identifying objects within 

30 a space, to classifying animals or other objects or materials according to a taxonomy or other classification scheme. 
For the sake of clarity, the diagnostic tester of the present invention will be specifically described hereinbelow in the 
context of fault diagnosis of complex systems, wherein malfunctioning systems having a plurality of f unctbnal compo- 
nents or elements are analyzed to isolate the source of a fault. 

Referring to Fig. 7, the basic preferred functional configuration of processor 17a, CPU 24, controller and diagnostic 

35 test signal analyzer 32, and the control portion of sensor/control unit 42 and diagnostic tester 44 comprises a controller 
section 52, a test signal evaluation section 54, and a sufficient testing evaluation section 56. Controller sectbn 52 
generates control signals Scon controlling the associated display 1 2, 22, 36 and/or test generator 34 and system A. 

Test signal evaluating section 54 evaluates inputted test signal data resulting from testing or sensing of some 
aspect, referred to hereinafter as a parameter, of system A, and produces a plurality of n candidate signals Sq corre- 

40 sponding to n possible diagnoses of. or diagnostic conclusions about the condition of system A, where r^2, and each 
candidate signal Sq has an associated level of certainty of representing a valid diagnosis, that is, a diagnosis having 
associated therewith an acceptable level of confidence that the diagnosis is correct. Hereinafter, the candidate signals 
having the first and second highest levels of certainty of being valid are denoted Scpi and Scp2. Test signal evaluating 
section 54 also produces first and second certainty signals Sp^ and Sp2 corresponding respectively to measures or 

45 values of the first and second highest levels of certainty associated with candidate signals S^p^ and Scp2- Test signal 
evaluating section 54 also preferably produces uncertainty signals corresponding to a measure of the uncertainty 
that the evaluated test signals Sj can be validly evaluated to produce a diagnosis. 

Sufficient testing evaluating section 56 produces an output signal S3 indicative of whether sufficient test data has 
been evaluated to declare a diagnosis based on at least the certainty signals Sp^ and Sp2. and preferably also based 

so on uncertainty signal Sy. In addition, test signal evaluating section 54 advantageously also generates an associated 
signal Sf^ corresponding to a measure of the level of certainty that a conclusion other than Scpt* Scp2 and Sy Is the 
correct diagnosis, and the output signal Sg of sufficient testing evaluating section 56 Is also based on signal S|^. 

Referring to Fig. 8, in general, the procedure followed to make a system diagnosis in accordance with the present 
invention comprises th steps of selecting information sourc s (the diagnostic tests/sensed parameters) to be evalu- 

ss ated (step SSI); collecting Information via these informatbn sources (performing th sel ct dtest/s nsingthesel cted 
parameter and evaluating the results to produce test data) (step SS2); analyzing th information (test data) collect d 
(step SS3); drawing a candidate conclusion or conclusions about the situation examined (generating candidate signals 
Sc) (step SS4); and determining wheth r sufficient testing has been conducted to declare one or more of the conclu* 
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sions as the diagnosis, or to declare that further testing is unlikely to establish a diagnosis (Step SS5). If the step SS5 
determination is positive, then further testing is terminated and a diagnosis is declared, if appropriate (step SS6). If the 
St p SS5 determination is negative, then the procedure is r peat d by retuming to step SSI. unless some other ter- 
mination criterion is satisfied. 

5 In accordance with the present invention, any conventional method can be used to select the tests to be performed/ 

parameters to be sensed, and to collect the test data. Simllarty. any conventional method can be used to analyze the 
test data and draw candidate conclusions (generate candidate signals Sq) which also provide a measure of the levels 
of certainty that the various candidate conclusions are valid. 

A preferred model-based method of test selection and test data analysis for fault diagnosis of complex systems 

10 in accordance with the present invention which employs a modified form of Dempster-Shafter evidential reasoning will 
now be described. In accordance with this method, the problem of diagnosis is approached as a collection of subprob- 
lems: 

(1 ) Modeling : A diagnostic problem is treated as a modeling problem in which available Information sources, the 
IS desired set of diagnostic conclusions, and their Interrelationships are specified. 

12) Optimization : The information sources are selected for consideration in a way that optimizes the diagnostic 
process according to one or more criteria. 

(3) Logical Inference : Test data derived from the information sources is used to draw inferences about other infor- 
mation sources and about the diagnostic conclusions in the model. The inferences are drawn based upon the 

20 relationships specified In the model. 

(4) Statistical Qualification : Statistics describing the support and plausibility of each diagnostic conclusion in the 
model are computed based upon the Information sources examined, the information collected, and the confidences 
In the reliability of that information. 

(5) Statistical Inference: Statistics generated from collecting information from each information source are com- 
25 bined to determine the total support and plausibility for each diagnostic conclusion in the model. 

These statistics are then used to estimate the probabilities that the corresponding conclusions constitute correct 
diagnoses. The resulting probabilities are then examined in the manner described In more detail hereinafter to deter- 
mine If sufficient information has been collected to make a confident, valid diagnosis, and if not, whether further test 

30 data should be collected. If the determination is positive because sufficient information is deemed to have been col- 
lected, then further testing and evaluation are terminated and the diagnosis corresponding to candidate signal S^pi 
having the highest certainty level of being valid is declared as the diagnosis. Alternatively, if the determination is positive 
b cause the uncertainty due to conflict Is such that no confident valid diagnosis can be made In the near term, then 
further testing and evaluation are terminated and the diagnosis which is declared is that corresponding to uncertainty 

35 signal S^, i.e., that a valid diagnosis cannot be made. If the determination is negative because insufficient information 
has been collected, then further testing and evaluation are conducted until a positive determination is obtained, unless 
another termination criterion Is satisfied. Two such criteria are whether a trend analysis Indicates that additional test 
data collection is not Improving the ability to select a conclusion as the fault diagnosis; and whether all information 
sources have been used a predetermined number of times without a determination being made that sufficient infor- 

40 mation has been collected. Preferably, the repetition limit varies for each test according to the confidence values (de- 
scribed in more detail hereinafter) assigned to the test. In the preferred embodiment, the standard limit Is 4 unless the 
value of the greater of the Good and Bad base confidences is less than 0.85. Then the limit Is reduced as follows: 
if 0.76 < Base < 0.85, then the limit = 3; 
if 0.6 < Base < 0.75. then the limit = 2; and 

45 If Base < 0.6. then the limit = 1 ; 

where Base is the greater base confidence value. 
A typical fault Isolation determination using a diagnostic model generated by the aforementioned STAMP program 
involves choosing tests, performing the tests, evaluating the results of the tests to assign a value of "Good." "Bad." or 
'Unknown" to the results, and qualitatively qualifying the "Good" and "Bad" values according to the confidence of the 

so user In the result. (For example, the user can specify that the designated Good/Bad result Is certain, somewhat certain, 
marginal, somewhat uncertain, or uncertain.) The qualifiers are then converted to numerical confidence factors which 
are used to compute evkiential statistics (support and plausibility) for each conclusion provided In the model. The fault 
diagnosis process will now be described in detail with reference to Figs. 9-13. 

S5 THE DIAGNOSTIC MODEL 

Referring to Fig. 9, the first step (SS 10), is to define the diagnostic model. Preferably, the diagnostic model used 
consists of a description of the flow of informatbn through a system to be diagnos d. Th infonmatlon f tow Is repr sented 
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as a set of diagnostic tests (information sourc s), a set of diagnoses (conclusions to be drawn), and a set of relationships 
between the tests and the conclusions. In addition, the model preferably includes parameters for weighting the test 
sel ction optimization process, grouping tests and conclusions, and specifying types of tests to b performed (e.g. 
functional tests, conditional tests, or asymmetric tests). The model may be specified as a topological description of a 
s system, the dependency model between failure modes and probe tests, a collection of IF.. .THEN... rules as in a rule- 
based expert system, or any other formulation that describes the relationship between information sources and con- 
clusions. 

Advantageously, the tool used to develop the model is the aforementioned STAMP program, which is described 
in the ARINC Research publication. STAMP User's Guide , Tech. Note 213 (March. 1967). by the inventors Sheppard 
10 and Simpson herein, and which is hereby incorporated herein by reference. STAMP can be used to develop any model 
that meets the requirement described in the previous paragraph. The models generated by STAMP permit diagnoses 
to be performed in a dynamic, context sensitive environment. Diagnostic strategies are computed based on known 
information; and can be tailored and adapted to changing conditions, such as failing test equipment, or change of 
emphasis from minimum time to fault isolate to minimum required skill level to fault Isolate. 

75 

GEfst^RATING THE DIAGNOSTIC SEQUENCE 

Once a model has been established (STAMP creates a computer readable file which is loaded into the processor 
constituting controller and test signal evaluating sections 52 and 54), the user may. optionally, specify a collection of 

20 symptoms and initial conditions (such as weighting criteria and groupings). 

In order to provide an efficient and dynamic approach to diagnosis, two elements are essential: 1 ) a method of 
choosing information sources that result in a conclusion being reached utilizing a minimum anrK>unt of some required 
resource; and 2) a method for inferring information from actions taken by the user in order to adjust the diagnostic 
system's knowledge of the current state of the problem. The manner in which information sources are selected so as 

25 to optimize the diagnostic process in accordance with the present invention will be discussed next. The following dis- 
cussion assumes that no grouping has been specified, no test outcomes (symptoms) have initially been determined, 
and a cost weight has been selected for information source selection optimization (I.e.. the information source selection 
is biased in favor of a less expensive source (test) over a more expensive source). 

In the preferred embodiment a complete diagnostic sequence is generated in two test selection phases (step SS 

30 20). The first phase (steps 214-226 of Fig. 10) consists of selecting tests based upon the amount of information they 
provide (and subject to any other selected weighting or grouping criteria). When a diagnostic procedure Is first started, 
all of the tests in the model are available for consideration and all of the possible diagnostic conclusions in the model 
are still under consideration. If symptom information (or initial conditions) are then loaded by the user, they are used 
to prune the problem space using the inference meta-rules described below. At this point, some tests are still unknown 

35 and some conclusions have no evidence denying that they are possible. The tests that provide the most discriminating 
information about the candidate conclusion set regardless of the outcome of the individual tests are selected, as de- 
scribed in more detail hereinbelow. Referring to Fig. 9. when the selected test is performed (step SS30) and the results 
of a test are evaluated to produce test data (step SS40). the test data are processed by an inference engine, to be 
described hereinafter (step SS60), and another test is chosen in the same fashion as described above. Referring to 

40 Fig. 1 0, this process continues until there are no more tests to choose (i.e. all tests have either been evaluated or their 
outcomes have been inferred) (step SS210). 

Referring still to Fig. 10, the specific process of choosing a test is based on Shannon's Theory of Information. For 
each test that is available, the amount of Information that will be leamed If the test passes (I.e., Is good) Is computed 
(step SS214). The amount of information learned from a good outcome is taken to be the number of tests that the test 

4S depends on plus the test itself. This test passed information value is then multiplied by the test's corresponding "good" 
"base confidence", a measure of the maximum confidence one can have in a good outcome, to produce a first product 
value (step SS216). 

Then the amount of information that will be leamed if the test fails (i.e. is bad) is computed (step SS218). The 
amount of information learned for a bad outcome is taken to be the number of tests that depend upon the test, plus 

so the number of tests that can be designated "not needed," plus the test itself. A test may be designated as not needed 
if the test neither depends on nor Is depended on by the test under consideration unless designating that test 'not 
needed' creates a new ambiguity within the model. A new ambiguity Is created if two or more conclusions that used 
to have different test dependency sets would have the same test dependency set because a test which Is the only 
differentiator between the conclusions is the test about to be declar d not needed. 

SS Once computed, the test failed information value is then multiplied by the t st's corresponding 'bad' base confi- 

dence, a measure of the maximum confidence one can have in a bad outcome, to produce a second product value 
(step SS220). Base confidences are defined by the user for both good and bad lest outcomes. The two values do not 
have to be qua!. 
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Next, the minimum of the first and s cond product values is identified for each test (step SS222). Normalized 
weight values are then computed and multiplied by the minimums just stored to produce third product values. As with 
the base confidence values, cost and other direct weight valu s affecting test selection are also specified by the us r 
(or learned from actual diagnostic experience). Direct weighting of the tests is accepted which is inversely proportional 
to the informational value of the test, and indirect weighting of the tests is accepted through weighting of the related 
conclusions which is directly proportional to the value of the test being weighted. As will b appreciated by those skilled 
in the art. a direct weight is a weighting factor applied directly to the tests in the model. Examples of direct weights 
include test time, test cost, or skill level required to test. An indirect weight Win^i^ectUl a^pplied to the conclusions in a 
model are attributed to the tests as follows: 

W,„ J = Zi(Widi)/IiWi 

where 

Wj = , a user-specified indirect weight associated with conclusion C; 
tj = a test 

dj = 0; if tj does not depend on Cj 
d| = 1 ; if t| does depend on Cj. 



Examples of indirect weights include failure rate, criticality and safety 

It will also be appreciated that when one or more weights are applied to Information values in the test selection 
process, then an emphasis factor may also be associated with any weight which has the effect of increasing the sig- 
nificance of the weight over other weights and/or the information value. In additbn, if test groupings have been specified. 

2S th n three additional test selectbn features are available. First, time and cost penalties can be applied that affect time 
and cost weights until a test group is entered (a test within the group is selected). Second, specific sequencing of tests 
within groups can be defined which overrides normal test selection choices. Third, sequencing of entire test groups 
can be defined which also override normal test selection choices. 

The normalization process results in the weights for a given criterion summing to 1 .0. The maximum of the com- 

30 puted third product values is then determined (step SS226). The test with this maximum third product value is selected 
as the next test to evaluate. 

During the first phase of test selection, an optional additional form of test selection advantageously also is provided. 
In this test selection mode, the user can specify that the selection be based on a hypothesis (discussed in more detail 
hereinafter) selected by the user. If the user selects a hypothesis, then all choice overrides and modifiers are disabled, 
3S and a test is chosen based on the following rules: 

1 . A test that has a Bad outcome and depends upon the hypothesis and a minimum number of other unknown 
conclusions. 

2. A test that has a Good outcome and does not depend upon the hypothesis, but does depend upon a maximum 
^0 number of other unknown conclusions. 



The first of these rules is preferred over the second. 

At the point all of the tests have a value (either given or inferred), as determined in step SS210. the second phase 
of choosing tests begins. At this point, one or more candidate failures (conclusions) have been identified and are given 

^ as a current hypothesis, as described in more detail hereinbelow. The levels of certainty associated with these failure 
conclusions are processed in the manner described in more detail hereinbelow to determine if enough evidence has 
been gathered to declare a diagnosis. If a determination is made that more evidence is necessary, and testing should 
not terminate, then another test needs to be chosen. At this point, rather than selecting a test that maximizes the 
amount of information about the system, an attempt is made to choose a test that will either separate multiple hypoth- 

50 eses or further support a single hypothesis. The purpose of this test choice process is to identify sources of evidence 
that will increase the level of certainty that a selected candidate conclusion (i.e. the current leading hypothesis, as 
defined hereinafter) Is indeed the conclusion that should be drawn. 

A test is chosen based on the quantity of evidence according to a process which can include up to four evaluation 
steps. First, if more than one current hypothesis exists (step SS230), then a test is chos n that splits the top two 

55 hypotheses (th two hypotheses with the highest associated lev Is of certainty) (step SS232). This Is done by examining 
the dependencies and identifying which tests depend on one hypoth sis and not the other hypothesis. The t st that 
furnishes the most evidence and meets this criterion is selected (step SS234). If no tests exist that can separate 
hypotheses, or only one hypothesis exists, then the next evaluation step is performed. (The second and third evaluation 
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steps attempt to find a test that provides the most support for the leading hypothesis. In both of these steps, it is 
assumed that a Bad test outcome is being sought.) 

In the second valuation step, if only on hypothesis has been selected, then the t sts are evaluated to find the 
test that provides the most support for the hypothesis (step SS236). Similarly, if there are multiple current hypotheses, 

s and the first step failed to choose a test, the tests are evaluated to find the test that provides the most support tor the 
I ading one of the hypotheses, i.e.. the current hypothesis with the highest level of certainty associated therewith. The * 
intent here is to further strengthen the current or leading hypothesis, which should bring the testing sufficiency evalu- 
ation process closer to terminating the fault isolation process. Specifically only the tests that depend upon the current 
or leading hypothesis are considered. The test that denies the most conclusions other than the current/leading hypoth- 

10 esis is selected at this point (step SS238). To deny a conclusion, the test cannot depend on that conclusion. If no tests 
exist that support the current or leading hypothesis, then the next evaluation step is performed. 

In the third evaluation step, in the event that a test has not been chosen from step 2, then tests that deny the 
current or leading hypothesis are examined (step SS240). To do this, only the tests that do not depend upon the 
hypothesis conclusion are considered. The test that supports the most conclusions other than the hypothesis is selected 

IS at this point (step SS242). To support a conclusion, the test must depend on that conclusion. If no tests exist that deny 
the current/leading hypothesis, then some test must be selected, and the last evaluation step is performed. 

In the last evaluation step (SS244), in the event that no tests can be chosen that meet any of the above conditions, 
then the test with the fewest trials Is selected just to select some test. If this fails to select a test, then advantageously 
the user may be offered the opportunity to choose a test. Failing user selection, an end to testing is declared even if 

20 the testing sufficiency evaluation otherwise indicates additional testing is appropriate. 

INFERRING INFORMATION FROM TESTING 

Referring to Fig. 9. once a test is chosen, the next step is to evaluate the test outcome to provide the test data 
2S (Good/Bad with associated confidence factor, or Unknown) (steps SS40-50). The diagnostic model Is then analyzed 
using the hybrid inference engine to determine what additional information can be inferred from the new test outcome 
(step SS60). The Inference engine has two parts, one employing logical inference, and one empbying statistical infer- 
ence. The first, logical, part, which is only active during the first phase of fault isolation as described above, computes 
logical inferences based on the test outcomes according to i) the STAMP model assumption that there is a single fault 
30 or a limited number of specified multiple faults to be isolated and hence a single valid diagnostic conclusion; and ii) 
eight inference meta-rules (wherein a Good/pass outcome Is equivalent to 'false" in logic and a Bad/fail outcome is 
equivalent to "true"): 

1. IF a test t[j] is unknown, THEN no inference is available except t[j] is unknown. 
3S 2a. IF a test t[j] is false. AND test t[j] is not a negative inference test, as defined hereinafter, THEN every test upon 

which it depends is false. 

2b. IF a test t[j] is false. AND test t[j] is not a negative inference test. THEN every conclusion upon which it depends 
is false. 

3a. IF a test t[j] is true. AND test t[j] is not a positive inference test, as defined hereinafter. THEN every test that 
40 depends on t[j] is true. 

3b. IF a test t[j] is true, AND test t[j] is not a positive inference test. THEN every conclusion upon which t[j] does 
not depend is false (due to the single conclusion assumption noted hereinabove). 

3c. IF a test t[j] is true, AND test t[j] is not a positive inference test, THEN any test t[p] that is currently unknown 
and is not a dependency of t[j] is not needed, UNLESS declaring t[p] not needed creates a new ambiguity among 
^s the conclusions (due also to the single conclusion assumption). 

4. IF after rules 1*3 have been evaluated, any t[p] is unknown and depends on all of the unknown conclusions, 
THEN t[p] is true. 

5. IF after rules 1-4 have been evaluated, any t[p] is unknown and depends only on false conclusions. THEN t(p] 
is false. 

so 6. IF the outcome of test t[j] is linked, as defined hereinafter, to an outcome of t[l], THEN the outcome of t[i] is 

determined by the linkage, AND rules 1-5 are applied to t[i]. 

7. IF a test ty] is true, AND ty] is a negative inference test, AND t[j] is cross-linked, as defined hereinafter, to a 

positive inference test t(i], THEN t[i] is true. 

8. IF a test is false, AND t[j] is a positiv inference test. AND t[j] is cross-linked to a negative inference test t[i], 
ss THEN t[i] is false. 

For purposes of these tests, a negative inf rence test is defined to be a test that will result in Inference only if it has a 
true value. If th value of a n gative inf erenc test Is false, then no inference can be made except the value of the test. 
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A positive inference test, on the other hand, is defined to be a test that wilt r suit in inferenc only if it has a false valu . 
If the value of a positive inference test is true, then no inference can be made except the valu of the test. The linkage 
mentioned in rule 6 refers to the situation in which a test t[j] being true triggers th value of a linked test t[i] being false, 
and a test t[j] being false triggers the value of t(i] to be tru . The cross linkages mentioned in rules 7 and 6 mean that 

5 two tests are declared to be asymmetric (i.e. either positive or negative inference), and as such, a false value for a 
negative inferenc t st triggers a false valu in a linked positive inference test, and a true positive inference test triggers 
a true value in a linked negative inference test. 

It will be appreciated that other sets of inference rules, or other reasoning approaches may also be used to derive 
information from the test data for use in the diagnostic process. 

10 The second, statistical, part of the inference engine functions throughout the entire diagnostic process (and thus 

in parallel with the logical part during the first phase of test selection when the inferential part is active). Several statistics 
based on the evidence gathered are computed that describe the level of certainty for each diagnostic conclusion in 
the model that it is the correct conclusion. This computation incorporates a method of attributing evidence to conclusions 
in the model to compute measures of support and plausibility for the conclusions. As is known from Dempster-Shafer 

IS theory, there are two measures of the amount of evidence accumulated for a conclusion-"belief" and ■plausibility," 
They are directly related to the evidence provided by tests. Support evidence from a test affects the belief of a conclu- 
sion. Belief differs from support in that it represents the total support evidence available for a given conclusion: support 
is that portion of the evidence from one test that is directed to a given conclusion. The Supporty^i^Cj] value used below 
is analogous to belief. 

20 Denial evidence affects the plausibility of a conclusion. Just as belief is the accumulation of support, plausibility is 

the accumulation of denial. However, it has a different sense than denial. Plausibility represents the upper limit on the 

belief. The relation between belief and plausibility Is: 

If the belief In a proposition A Is equal to Pr(A) (the probability that A is true), the denial of proposition A is equal 

to the Pr(not A). 0<Pr(not A) and Pr(A) + Pr(not A)<1 .0. 
25 then Pr (A) < 1 -Pr(not A) 

or belief < plausibility 

Evidence for a proposition is represented by a credibility interval [belief, plausibility]. Each interval is a subinterval 
of the probability interval [0, 1 ] with the lower bound (the belief) representing the degree to whksh the evidence supports 
the hypothesis, and the upper bound (the plausibility) represents the degree to which the evidence fails to refute the 

30 hypothesis. Under Dempster-Shafer theory, the difference between belief and plausibility represents the residual ig> 
norance or uncertainty Under Bayesian theory, the total probability of a proposition and its negation must be one, while 
under Dempster-Shafer theory, these probabilities do not necessarily sum to 1. The difference is uncertainty. Thus, 
the support measure quantifies the amount of supporting evidence for a given conclusion and is given by the confidence 
In the test outcome. Plausibility, on the other hand, provides a measure of the denial of conclusions and is simply one 

3S minus the support of all conclusions not supported by the test outcomes. The advantage of a Dempster-Shafer approach 
to estimating levels of certainty over a Bayesian approach is that prior probabilities do not need to be determined. 
Rather, the support and plausibility measures are determined solely by testing, the confidences in the test outcomes; 
and the known relationships between tests and conclusions. 

The computations performed by the statistical part of the inference engine are based on a modified version of the 

40 Dempster-Shafer evidential reasoning process. This process associates a confidence value with each test outcome 
in the range of 0.0 ... 1 .0. As noted hereinabove, base confidences define maximum confidence values for good and 
bad outcomes respectively. In addition, the user specifies if qualifiers are to be associated with the good and bad 
outcomes. If qualifiers are not associated, then the base confidences are selected as default confidence values, e.g., 
0.999, which advantageously can be either predetermined independently of the diagnostic model or can be specified 

^ for a particular model at the time the model Is being developed. If qualifiers are associated, then the user (optionally) 
lists the appropriate qualifiers in decreasing order of confidence level. Othenvise, default qualifiers, e.g.. certain, some- 
what certain, marginal, somewhat uncertain, uncertain, are used. Once the qualifiers are associated, good test and 
bad test confidence values are assigned to the qualifiers as follows: 

so 

Confidences^] = {)^[(N-i)/(N-1)]*Base Confidence} + 0.5 

where 

ss ConfidenceltJ = the good/bad confidence valu for th good/bad outcome of test t,^ 
Base Confidence = the corresponding good or bad bas confidence assigned to the test 
N = the total number of qualifiers (N>2) 

i = the ordinal number of the selected qualifier 
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For example, given the default qualifiers and a good base confidenc of 1.0. good t st confidence values would be 
assigned as follows: 



certain 


- 1.0 


somewhat certain 


- 0.875 


marginal 


- 075 


somewhat uncertain 


"0.625 


uncertain 


- 0.5 



At the outset of the statistical computations, the total accumulated support and denial values for each conclusion. 
Supportj^jtailcJ Denial-Totajlcj. respectively, and the uncertainty signal Sy are each Initialized to 0.0; and the plau- 
sibility value for each conclusion. Plausibility[Cj], and the Uncertainty value are each initialized to 1 .0. If a test outcome 
is true, then that test outcome supports conclusions upon which it depends and denies conclusions upon which it does 
not depend. On the other hand, if the test outcome is false, then that test outcome supports conclusions upon which 
it does not depend and denies conclusions upon which it depends. The amount of support tor a given conclusion i by 
a given test ti, outcome is computed as follows: 



20 



where 



Support[Cj] = Confidence[t,^]/ICI 



2S 



30 



C = set of conclusions supported 

IC1= size of C. 

Denial[Cj] = Confldence[t|J 

The above equations apply only for computing support and denial from a single source of evidence. Evidence 
furnished by multiple Information sources are combined using a modified form of Dempster's Rule of Combinations. 
First a measure of internal conflict K is computed: 



K = I|Ij(Support^^^,[Ci]* SupportlCjl; loj. 

35 Next, the number of conflicts is tallied. A conflict occurs when the outcome of a test denies all of the current hypotheses, 
which hypotheses correspond to candidate signals Sq. and are determined In the manner described hereinbelow. 

To determine the current hypothesis, all of the tests must have a value (good, bad, unavailable, or unneeded). At 
this point in the diagnosis procedure, the first phase of the test selection process has been completed and the second 
phase has been commenced. Since most users are at least Intuitively familiar with probability preferably the first step 

40 In determining the current hypothesis (or hypotheses) is to compute an estimate of the probability that each conclusion 
Is the answer The evidential support and plausibility statistics for each conclusion are converted to an estimated prob- 
ability value Sp[ci] (signal Sp) as follows: 



4S 



so 



55 



Sp[c.] =J^(Support^^^,[Cj) + Plauslbi!ity[Cj)). 

The estimated probability of the UR, defined hereinafter, Is treated as a special case by assuming its estimated prob- 
ability is strictly the value of Support[U] (the support for the UR calculated as set forth hereinbelow. which corresponds 
to signal Sy). This is because PlaustbilityfU] for the UR is always 1 .0. which would unacceptably force the estimated 
probability to be at least 0.5 regardless of whether any conflict has been encountered. (Although this creates potential 
problems when values are normalized, since different types of values are being normalized, the ability of the diagnostic 
tester to determine the sufficiency of testing does not appear to be adversely affected.) The conclusion with the highest 
probability is automatically considered a hypothesis (signal Sqp^). A determination is then made whether any other 
conclusions have probabilities close enough to the maximum to be considered a hypothesis as well. A th reshold function 
is used to make this determination: 

If Sp,<0.1. then Threshold = 0.5 * Sp^ 

If Spi>0.5, then Threshold = 0.9 * Sp^ 

Else Threshold = Sp^ * (0.4 + Sp^). 
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If any conclusion has a probability value that xceeds the threshold, then that conclusion Is also considered a hypothesis 
(signal Scpn)^ 

In accordance with the present inventbn, the Dempster-Shafer method is modifi d so that direct conflict betw en 
test outcomes and hypoth ses results in an overall increase in uncertainty in th system. This modification entails the 
s introduction of a new diagnostic conclusion, referred to herein as the 'unanticipated result' (UR). The UR is never 
denied, and It is supported only In the event of a conflict. The current support value for the UR. I.e.. signal Sy, Is 
computed as follows in the absence of conflict: 

Sg = Sy •(Tests - 1 yTests. 

In the event of a conflict, a value is calculated as above, and an additional computation Is made using this value to 
derive the Sy signal value used to determine the sufficiency of the testing: 

Sy = Sy + (Conf licts*K*Confidence[t,^iyrests 

where 

20 Conflicts = the number of times a conflict is encountered 

Tests = the number of tests performed and actually evaluated so far (does not include tests for which outcomes 

have been inferred; each repetition of the same test is counted) 

The new current total support for each conclusion in the model. SupportTQta|[Cj], based on the accumulated test data 
2S currently inputted. Is then computed: 



SupportTota|[Cil = (SupportTotoi[C|] * Support(Cjl + 
Uncertainty * Support[C|] + 
Support^^j^,[Cj] * (1-Confidence[t^]))/(1-K). 

If Support[Ci] is less than 1 /Conclusions, where the Conclusions value is the number of conclusions in the diagnostic 
3S model, then Support[Cj] is reduced by 25% to gradually remove the conclusion from consideration. Next, a new uncer- 
tainty value Is computed: 



Uncertainty = (Uncertainty * (I-Confidence(lj))/ (1-K). 

40 

Finally, the support values Support[Cj] are normalized as indicated above. Each total support value is nornnalized 
to sum to 1 .0 minus the support for the UR: 

^5 * SupportTpto|[Ci]=SupportT^j3,(Ci] * (1 -Sy yz^SupportT^tailCj] 

In addition to computing the support values for the conclusions in the model, the denial values must also be com- 
puted. Denial is computed as follows: 

DenialT^^^[c,l = Denial^^^ICj] + DeniallCj]. 



From the denial computatbn, plausibility is computed: 

Plausibility[CjJ = 1 - (Denialy^^[Cj]/Tests). 
To summarize the computational flow, as shown in Fig. 11 , th first step (SS600) is to initializ the values for total 
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support and denial of conclusions, plausibility and uncertainty, support of th UR; and to define the initial current hy- 
potheses. (All of the possible model conclusions advantageously are selected as the initial hypotheses.) The next step 
(SS602) is to perform a test s lect d as describ d hereinabove, and evaluate the outcome. Then. (SS604), a confidence 
value is assigned to the test outcome; and the support and denial for each conclusion are calculated based on the t st 

5 outcome confidence value (step SS606). The n xt step (SS608) determin s if the test outcome conflicts with all current 
hypotheses. Then the measure of internal conflict K is obtained (SS610) based on the old total support values and the 
present test outcome support values for all of the conclusions. The present value of the UR support is then computed 
based on the number of tests evaluated, if the present test outcome is not a conflict; and based on the number of tests 
evaluated and the number of conflicts, if the present test outcome is a conflict (SS612). Next (SS614). the new total 

10 support value for each conclusion is computed based on the old total support value, the present test outcome support 
value and confidence value, the old uncertainty, and K. Then a new uncertainty value is computed based on the old 
uncertainty value, the present test outcome confidence value and K (SS616). The next step (SS618) is to normalize 
the total support value for each conclusion based on the old total support value, and the present UR support value. 
Then (SS620). a new total denial is computed for each conclusion based on the old total denial and the present test 

IS outcome denial value. Next (SS622), the present plausibility is computed for each conclusion based on the present 
total^denial and the number of tests evaluated. The current hypothesis/hypotheses (signals S^p) is/are selected 
(SS624) based on the present total support and the plausibility for each conclusion. Finally (SS626), the sufficiency of 
testing is evaluated based on values derived for the two highest levels of certainty, Sp^ and Spg. and the value of the 
UR support, Sg, in the manner to be described in more detail next, to determine whether to continue or terminate further 

20 testing, and whether to declare a diagnosis. 

TERMINATING THE DIAGNOSTIC PROCESS 

The testing sufficiency evaluation approach of the present invention operates to recognize when sufficient differ- 

25 entiation between levels of certainty regarding the validity of altemative candidate diagnoses exists. In the preferred 
Implementation described herein, a neural network is trained to recognize patterns in the levels of certainty Indicative 
of whether a diagnosis can be made or there is so much conflict that the diagnosis process should be halted. Further, 
the preferred neural network is configured to be generic, that is, it can be used without modification or retraining for 
different diagnosis (classification) problems and different diagnosis models. 

30 Referring to Fig. 12, a neural network 100 according to the present invention preferably comprises a set 110 of at 

least first, second, third and fourth input nodes 111, 112, 113 and 114. respectively. Each of the input nodes has first, 
second and third outputs weighted by a predetermined fixed weight. Input nodes 111 and 112 respectively receive 
certainty signals Sp^ and Sp2 as an Input. Input node 1 1 3 preferably receives uncertainty signal Sy. but this input need 
not be present; and input node 114 is a bias node receiving a predetermined, e.g.. unity, bias signal. 

3S If desired, additional input nodes can be provided which receive additional certainty signals corresponding to the 

successive next highest levels of certainty associated with the model conclusions. Such additk^nal certainty signals 
can also include signal S^g referred to hereinabove corresponding to the level of certainty that some conclusion other 
than the model conclusions and the UR is the correct diagnosis. However, for the disclosed preferred embodiment, 
additional input nodes are not necessary. Since the testing sufficiency determination does not depend on the character 
of the conclusbns constituting the leading hypothesis, only one node is needed corresponding to the highest level of 
certainty (estimated probability value in the preferred embodiment). In addition, since only a single failure is being 
isolated by any given analysis, it is only necessary to compare the highest probability value with the next highest in 
order to determine whether the highest is "high enough.' 

Neural network 100 further comprises at least one set 120 of first, second, third and fourth intermediate nodes 
121, 122, 123 and 124 each having an output weighted by a predetermined fixed weight. Intermediate node 111 receives 
the sum of the first weighted outputs of input nodes 110 as an input, intermediate node 122 receives the sum of the 
second weighted outputs of input nodes 11 0 as an input, intermediate node 1 23 receives the sum of the third weighted 
outputs of input nodes 110 as an input, and intermediate node 124 Is a bias node like node 114 receiving the prede- 
termined bias signal. At least one intermediate layer is preferred, but not necessary. Fig. 13 shows a viable two layer 

so neural network which has been implemented. 

The always active bias nodes provided in the input and intermediate layers are generally required to ensure that 
there is always some activity coming into each node of the upper layers, and to prevent singularities. 

Neural network 100 also comprises a single output node 130, responsive to the sum of said weighted outputs of 
intermediate nod s 120, for producing output signal S3 indicative of whether sufficient test data has been evaluated 

55 to declare a diagnosis. Only a single output nod is needed for th preferred embodiment becaus only of simple 'yes/ 
no' determination n eds to be made. 

The output weights of input and intermediate nodes 110 and 120 correspond to a model of expert data correlating 
combinations of at least certainty signal values, and pref rably certainty and uncertainty signal values, with at least 
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one expert opinion regarding the ability to declare a diagnosis on the basis of each combination of such signal values. 
The expert data model can either be specific to a type of system being diagnosed, or as will be described in more detail 
hereinafter, can be gen ric to systems in general. 

The node output weights advantageously are obtained by training a backpropagation neural network having the 
5 same architecture as said neural network using empirical certainty and uncertainty signal training data obtained from 
testing of real systems, and conclusions derived from at least one expert regarding the sufficiency of th information 
represented by the combinations to declare a diagnosis for a specific system to be tested. 

Alternatively, and preferably, the backpropagation neural network is trained using normalized randomly generated 
certainty and uncertainty signal training data, and conclusbns derived from at least one expert regarding the sufficiency 
10 of the information represented by said combinations to declare a diagnosis lor systems in general. 

Preferably, the estimated probability values of the certainty and uncertainty signals are applied as the inputs to 
nodes 111 and 112. The training data includes first, second and third unambiguous combinations in which the data 
sufficiency determination is obvious. Advantageously, one unambiguous combination comprises certainty signal Sp^ 
having a probability value corresponding to 1 00% and certainty signal Spg and uncertainty signal Sy having zero values. 
IS Another unambiguous combination advantageously comprises certainty signal Sp^ and uncertainty signal each 
haviog a probability value corresponding to 100%, and certainty signal Sp2 having a zero probability value. The third 
unambiguous combination advantageously comprises both of the certainty signal Sp^ , Sp2 probability values having 
uniform probabilities and the uncertainty signal Sy probability value having a zero value. 

The neural network training data preferably also includes two data sets each having a predetermined number of 
20 ambiguous combinations wherein: 

both data sets have certainty signal probability values within a closed interval [0, 1 ] generated at random according 
to a lognormal distribution; 

a first one of the data sets has uncertainty signal probability values generated at random according to a lognormal 
2S distribution; 

a second one of the data sets has uncertainty signal probability values set to zero; 

the values so generated are adjusted by normalizing the certainty signal probability values in each ambiguous 
combination to sum to 1 - S^; 

all normalized certainty signal probability values and the uncertainty signal probability values less than or equal 
30 to a predetermined value based on the number of elements in the system are set to zero; 

the resultant certainty and uncertainty signal values in each of the ambiguous combinations are normalized to sum 
to the value one; and 

the larger of the resultant normalized certainty signal probability values is designated as the value Input to node 
111 in each of the ambiguous combinations where the certainty signal probability values are unequal. 

3S 

Preferably, the sum of the weighted outputs inputted to each of the intermediate nodes 120 and to output node 
130 is processed in the respective node with a sigmoidal activation function, which preferably is also logistic, and 
preferably the function 1/(1 +e**). where x is the unweighted output of a given node. 

An exemplary set of training data and the survey data from which the training data was derived are shown in the 
40 following charts; 



so 



NEURAL NETWORK TRAINING DATA 


Case 


Node 1 


Node 2 


Node 3 


Answer 


1 


1.00 


0.00 


0.00 


1.00 


2 


0.50 


0.00 


0.50 


1.00 


3 


0.50 


0.50 


0.00 


0.00 


4 


0.47 


0.30 


0.23 


1.00 


5 


0.59 


0.41 


0.00 


0.00 


6 


0.50 


0.50 


0.00 


0.00 


7 


0.39 


0.34 


0.27 


1.00 


B 


0.64 


0.36 


0.00 


1.00 


9 


0.58 


0.42 


0.00 


0.00 


10 


0.52 


0.48 


0.00 


0.00 


11 


0.32 


0.22 


0.46 


1.00 


12 


0.76 


0.24 


0.00 


1.00 
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(continued) 



NEURAL NETWORK TRAINING DATA 



Case 


Nod 1 


Node 2 


Node 3 


Answer 


13 


0.51 


0.49 


0.00 


0.00 


14 


0.51 


0.49 


0.00 


0.00 


15 


0.56 


0.44 


0.00 


0.00 


16 


0.67 


0.33 


0.00 


1.00 


17 


0.66 


0.34 


0.00 


1.00 


18 


0.58 


0.42 


0.00 


0.00 


19 


0.72 


0.28 


0.00 


1.00 


20 


0.55 


0.45 


0.00 


0.00 


21 


0.70 


0.30 


0.00 


1.00 


22 


0.52 


0.48 


0.00 


0.00 


23 


0.67 


0.33 


0.00 


1.00 


24 


0.27 


0.25 


0.48 


1.00 


25 


0.50 


0.50 


0.00 


0.00 


26 


0.52 


0.48 


0.00 


0.00 


27 


0.62 


0.38 


0.00 


1.00 


28 


0.14 


0.11 


0.76 


1.00 


29 


0.43 


0.27 


0.30 


1.00 


30 


0.68 


0.32 


0.00 


1.00 


31 


0.50 


0.25 


0.25 


1,00 


32 


0.53 


0.47 


0.00 


0.00 


33 


0.51 


0.49 


0.00 


0.00 


34 


0.54 


0.46 


0.00 


0.00 


35 


0.63 


0.37 


0.00 


1.00 


36 


0.38 


0.32 


0.30 


1.00 
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10 



IS 



20 



25 



30 



40 



SURVEY 

ANSWER 

Clci£2c4c5c6c7c8c2 clO ys 
.47 .30 .23 

.22 .36 .16 .25 

.40 .19 .40 

.39 .34 .27 

• 17 .41 .23 .19 

.19 .17 .27 .37 

.32 .21 .18 .29 

.11 .11 .14 .21 .14 .30 

.64 .16 .20 

.29 .36 .16 .20 

.17 .21 .22 .19 .20 
.23 .24 .20 .19 .15 

.44 .26 .30 

.33 .22 .26 .19 



.21 .18 .18 .42 

.20 .26 .16 .19 .19 

.16 .35 .18 .14 .17 

.23 .25 .17 .34 

.22 .21 .18 .24 .16 

.11 .13 .21 .12 .25 .17 

^ .15 .50 .19 .16 

.39 .24 .18 .19 

.17 .20 .20 .24 .19 



.23 .54 .22 

.13 .13 .18 .20 .22 .14 

.16 .35 .18 .14 .17 
.52 .26 .22 

^ .17 .41 .23 .19 



so 



55 
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ANSWER 

Ci£a^c4c5c6c2c8c2 clO UR 

•^^ -21 .23 .40 

•1'' -22 .11 .15 .20 ,14 

•15 .25 .25 .18 .16 

.52 .48 

•21 .39 .24 .16 

•10 .08 .06 .07 .06 .07 .56 

•15 .50 .19 .16 

IS .16 .18 .16 .29 .20 

•25 .21 .53 

•25 .50 .25 



10 



20 



•39 .24 .27 

.25 .40 .35 
•l'^ -19 .25 .16 .24 

.16 .45 ,39 

25 .16 .41 .24 .21 .14 

• 19 .31 .26 .24 



30 



Also shown in the training data are the answers selected according to the answers provided by the polled experts 

regarding the ability to make a confident diagnoses based on the probability value combinations. Since it was desired 
3S that neural network 100 be conservative in making a termination determination, it any expert indicated that a combi- 
nation warranted termination, that answer was selected as the answer to be used in training the network. It will be 
appreciated that the survey included combinations containing ten probability values plus a UR probability value. The 
probability values were generated as indicated above, with all values less than or equal to 0.1 being set to 0.0. The 
surveyed experts were engineers who were familiar with and had used STAMP. The experts were Instructed to assess 
40 the sun/ey combinations on the basis of the following assumptions: 

1. A system that has failed is being tested based on a diagnostic model. 

2. The Cf, numbers represent probabilities derived from the testing that a unit(Cp) within the system is the failure. 

3. The failure events are independent and a testability analysis has shown that each of the conclusions listed is 
^5 reachable and unique in a perfect information environment. 

4. The UR value represents the probability that something has occurred in the system that the model currently 
cannot account for. 

The training data set forth hereinabove has been applied to a backpropagatlon neural network having random 
so initial output weight values and trained using a conventional backpropagatlon training program wherein each combi- 
nation of certainty and uncertainty signal values was repeatedly applied to the corresponding input nodes of the neural 
network. An error value was computed folbwing each application corresponding to how much the actual output of node 
1 30 differed from the expected output for the combinatbn as defined by the selected expert answer in the training data. 
Specifically, the error was computed as the squared error: 

55 
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75 



20 



r= 1/2 (ri'"-oD* 

p-0 i-o ^ ' 



wher 



E = the measure of squared error 

Hp ^ the number of patterns in the training set 

/o n^ = the number ot output nodes 

p = the current input pattern 

i = the current output/target node 

TjtP) = the expected output ot node i given pattern p 

Oj(P) = the actual output of node i given pattern p 



from the value E, a correction term is computed which is then propagated bacl<ward through the network, adjusting 
the node weights as it goes. The correction term is computed from the partial derivative of the error with respect to the 
weights. The weight adjustment occurs as follows: 

p-0 



2S where 

Wjj('<)(t) = the weight from node j to node i at time t for layer k 

r| - the learning rate 

Oj<P *<-^) = the output of node j for combination p at layer k-1 

30 k = the current layer 

p = the current pattern 

np = the number of patterns 

8j(P*<) = the correction term for node I at layer k resulting from pattern p 

a = momentum coefficient 

3S AWjj(i<)(t) = change in the weight at time t. 

The correction term, 5j(P **), is computed as follows: 
Case 1 : (the output layer) 



40 



4S 



5i<P*> = (T*»-0/''''>)0**>(1-0<''''') 



Case 2: (all hidden layers) 



^•0 



so Although the use of a backpropagation neural network is preferred, other types of neural networks can be used. 

For example, counterpropagation as developed by Robert Hecht-Nielson provides essentially the same functionality 
with a different architecture. Hopfield networks. Adaline networks, and other supervised learning networks could also 
be used. 

As the training data is repeatedly present d to the n ural network, the network weights are modified gradually, 
55 folbwing the error surface defin d by the network weight space down the gradient to a local minimum. When the 
n twork settles, it contains a solution to the mapping problem. 
The node output weights which resulted from this training are: 
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the weights of the first, second and third outputs of input node 111 are (-9.640). 5.049. and (-1 .624). resp ctrvely; 
the weights of the first, second and third outputs of input node 112 are 15.641. (-8.491). and 2.436, respectively; 
the weights of the first, s cond and third outputs of input node 11 3 are (-6.680), 3.627. and (-1 .237), resp ctiv ly; 
the weights of the first, second and third outputs of input node 114 are (-0.422), 0.442. and (-0.427). respectively; 
5 and 

the weights of the outputs of Intermediate nodes 121. 122. 123 and 124 are. respectively, (-19.214), 11.457. (- 
2.483) and 4.081. 

This collection of weights has proven to make testing sufficiency determinatbns which correlate highly with the 
10 opinions of experts exposed to the same data. Specifically, neural network 100 has been tested with a set of 2601 
randomly generated test data representing all permutations of three numbers to two decimal places under the following 
constraints: 

1. The numbers were all in the closed interval [0,1]. 
IS 2. The numbers all summed to 1 .0. 

3. The first number was always greater than or equal to the second number. 

Twenty of these cases were randomly selected as a test survey and submitted to the same group of five experts 
used to generate the training data. On average, 4.25 of the experts agreed with neural network 1 00 on each test case. 
20 In one case, neural network 100 indicated termination was not appropriate, while four of the experts said it was appro- 
priate. This case was at the decision boundary of the neural network, and could have been decided either way. Thus, 
overall, the performance of neural network 100 was in agreement with the experts. 

It will be appreciated by those skilled in the art that the invention has been described with respect to exemplary 
embodiments, and that numerous modifications can be made therein without departing from the scope of the invention. 

25 

Claims 

1. Automatic test data evaluation apparatus for use in a diagnostic tester that declares a diagnosis on the basis of 
30 localisation or isolation of failed system elements of a system having a defined number of functional elements, 

said apparatus comprising: 

means for selecting the next test to be performed on the system to be tested; 
means for generating the corresponding test data and supplying them to the system to be tested; 
35 means for sensing and collecting predetermined system operating parameters; 

evaluating means (54), responsive to at least one inputted test signal corresponding to test data Sy relating 
to at least one predetermined parameter of the system, for producing using a probabilistic evidential reasoning 
method: 

40 first (Sqp^) and second (Scp2) candidate signals corresponding respectively to first and second possible 

diagnoses of the condition of the system respectively having the first and second highest levels of certainty 

of being valid, and 

first (Sp,) and second (Sp2) certainty signals corresponding respectively to values of said first and second 
highest levels of certainty; and 

45 

characterized by testing sufficiency means (56), responsive to said first and second certainty signals, for 
producing an output signal (S3) indicative of whether sufficient test data has been evaluated to declare a diagnosis, 
said testing sufficiency means comprising a neural network having an output node (130') and at least three input 
nodes (1 1 1 M 1 2*. 1 1 4') respectively receiving said first and second certainty signals and a predetermined bias signal 
so as an Input, said input nodes having weighted outputs, the output weights corresponding to a model of expert data 

correlating combinatbns of certainty signals with at least one expert opinion regarding the ability to declare a 
diagnosis on the basis of each combination of certainty signal values. 

2, The apparatus of claim 1 wherein: 

55 

said evaluating means also produces an unc rtainty signal Sy corresponding to a measure of the uncertainty 
that the evaluated at least one test signal can be validly valuated; and 
said neural network comprises: 
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a set (110) of first (111). s cond (112), third (113) and f urth (114) input nodes each having first, second 
and third outputs weighted by a predetermin d fixed weight, said first (111) and second (112) input nodes 
r spectively receiving said first (Sp^) and second (Sp2) certainty signals as an input, said third input node 
(113) receiving said uncertainty signal {S^^), and said fourth input node (114) rec iving said predeternnlned 
bias signal (BIAS); 

a set (120) of first (121), second (122). third (123) and fourth (124) intermediate nodes each having an 

output weighted by a predetermined fixed weight, and said first intermediate node (121) receiving the sum 
of the first weighted outputs of said Input nodes as an input, said second intermediate node (1 22) receiving 
the same of the second weighted outputs of said input nodes as an input, said third intermediate node 
(123) receiving the sum of the third weighted outputs of said input nodes as an input, and said fourth 
intermediate node (124) receiving said predetermined bias signal (BIAS); and 

an output node (130), responsive to the sum of said weighted outputs of said intermediate nodes, for 
producing said output signal (S J, the output weights of said input and intermediate nodes corresponding 
to a model of expert data correlating combinations of certainty and uncertainty signal values with at least 
one expert opinion regarding the ability to declare a diagnosis on the basis of each combination of certainty 
and uncertainty signal values, said testing sufficiency means thereby also evaluating said uncertainty 
signal to produce said output signal. 

The apparatus of claim 2 wherein: 

said evaluating means also produces a signal Sf^ corresponding to a measure of the level of certainty that a 
diagnosis not corresponding to said first or second candidate signals or to said uncertainty signal is a valid 
diagnosis; and 

said testing sufficiency means also evaluates said signal S|«j to produce said output signal. 

The apparatus of claim 2, wherein said evaluating means (54) comprises: 

means for storing a model of predetermined information flow relationships within the system under test; 
means for associating said test signals with sets of possible causes in accordance with said model, and for 
ascertaining the intersections of said sets of possible causes for combinations of test signals, said candidate 

signals being derived from said intersections; and 

means for determining for each candidate signal a measure of the level of certainty that it represents a valid 
diagnosis. 

The apparatus of claim 4, wherein said evaluating means (54) comprises means for determining the degree to 
which a conflict exists between at least said first candidate signal and said at least one test signal indicative of the 
level of certainty that an unanticipated result (UR) has been obtained, said uncertainty signal having a value cor- 
responding to said UR level of certainty. 

The apparatus of claim 1 wherein the values of said first and second highest levels of certainty are probability 
values. 

The apparatus of claim 2, wherein said uncertainty measure has a probability value. 

The apparatus of claim 5. wherein said UR determining means comprises: 

means for defining a plurality of conclusions corresponding to possible diagnoses of the system; 
means for assigning a confidence value to each successively inputted test signal; 

means for calculating the support and denial for each conclusion based on said confidence value of the cur- 
rently inputted test signal; 

means for determining if the currently inputted test signal conflicts with a then selected set of said conclusions 
corresponding to candidate signals; 

means for obtaining a measure of Internal conflict based on accumulated total support values and the current 
test signal support values for all of said conclusions; 

means for computing the present value of said uncertainty signal based on the number of t st signals evalu- 
ated, if the present test signal is not a conflict; and based on the number of test signals evaluated and the 
number of conflicts, if the present test signal is a conflict; 

means for computing a new total support value for each conclusion based on the old total support value, the 
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present test signal support value and confidence value, an old uncertainty value, and said measur of internal 
conflict; 

means for computing a n w uncertainty value based on the old unc rtainty value, the present test signal 
confidence value and said measure of int rnal conflict; 
s means for normalizing the total support value for each conclusion based on the old total support value, and 

the present uncertainty signal value; 

means for computing a new total denial for each conclusion based on an old total denial and the present test 
signal denial value; 

means for computing the present plausibility for each conclusion based on the present total denial and the 
10 number of test signals evaluated; 

means for selecting the current set of candidate signals based on the present total support and the plausibility 
for each conclusion. 

9. Apparatus according to claim 2. wherein said output weights of said input (111*114) and intermediate (121-124) 
IS nodes have values obtained by training a backpropagation neural network having the same architecture as said 

neural network using empirical certainty and uncertainty signal training data obtained from testing of real systems, 
and conclusions derived from at least one expert regarding the sufficiency of the information represented by said 
combinations to declare a diagnosis for a specific system to be tested. 

20 10. The apparatus of claim 9. wherein said output weights of said input (111-114) and intermediate (121-124) nodes 
have generic values obtained by training a backpropagation neural network having the same architecture as said 
neural network using normalized randomly generated certainty and uncertainty signal training data, and conclu- 
sions derived from at least one expert regarding the sufficiency of the information represented by said combinations 
to declare a diagnosis for systems in general. 

2S 

11. The apparatus of claim 5, wherein the values of said certainty and uncertainty signals correspond to probability 
values. 

12. The apparatus of claim 11, wherein said training data includes first, second and third unambiguous combinations 
30 in which the data sufficiency determination is obvious, said first unambiguous combination comprising said first 

certainty signal having a value corresponding to 100% probability and said second certainty signal and said un- 
certainty signal having zero values, said second unambiguous combination comprising said first certainty signal 
and said uncertainty signal each having a value corresponding to 1 00% probability and said second certainty signal 
having a zero value, and said third unambiguous combination comprising both of said certainty signals having 
35 values corresponding to uniform probabilities and said uncertainty signal having a zero value. 

13. The apparatus of claim 1 2, wherein said training data includes two data sets each having a predetermined number 
of ambiguous combinations wherein: 

40 both data sets have certainty signal values within a closed inten/al [0.1] generated at random according to a 

lognormal distribution; 

a first one of said data sets has uncertainty signal values generated at random according to a lognormal 
distribution; 

a second one of said data sets has uncertainty signal values set to zero; 
45 the values so generated are adjusted by normalizing the certainty signal values in each ambiguous combination 

to sum to (1 - Su), all normalized certainty signal values and the uncertainty signal values less than or equal 
to a predetermined value based on the number of elements in the system are set to zero; and then the resultant 
certainty and uncertainty signal values in each of said ambiguous combinations are normalized to sum to the 
value one; and 

so the larger of the resultant normalized certainty signal values is designated as the first certainty signal value In 

each of said ambiguous combinations where the certainty signal values are unequal. 

14. The apparatus of claim 11. wherein: 

5^ the sum of the weighted outputs inputted to each of the intermediat nodes and the output mode is process d 

in the resp ctive nod with a sigmoidal activation function; 

the weights of the first, second and third outputs of the first input node ar (-9.640). 5.049, and (-1.624), 
respectively; 

! 
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the weights of the first, second and third outputs of the s cond input node are 15.641. (-8.491). and 2.436. 
respectively; 

the weights of the first, s cond and third outputs of the third input node are (-6.680), 3.627, and (-1.237). 

r sp ctively; 

5 the weights of the first, second and third outputs of the fourth input node are (-0.422), 0.442. and (-0.427). 

respectively; 

the weight of the output of the first intermediate node is (-19.214); 
the weight of the output of the second intermediate node is 11 .457; 
the weight of the output of the third intermediate node is (-2.483); and 
10 the weight of the output of the fourth intermediate node Is 4.081 . 

15. The apparatus of claim 14. wherein said activation function is logistic. 

16. The apparatus of claim 15. wherein said activation function is 1/(1 -i-e'^). where x Is the unweighted output of a 
IS given node. 

17. A diagnostic tester for declaring a diagnosis on the basis of localisation and isolation of failed system elements of 
a system having a defined number of functional elements the tester comprising the apparatus according to any of 
the preceding claims, the tester further comprising input means (13,26.34,42,52) for supplying test signals to said 

20 evaluating means. 

18. The diagnostic tester of claim 17, wherein said input means comprises operator actuated input means (13,26) for 
entering test data relating to at least one predetermined aspect of the system, and for generating test signals 
corresponding to the test data. 

2S 

19. The diagnostic tester of claim 17, wherein said input means comprises automatic test means (34) responsive to 
a control signal for performing selected tests on the system, and for generating test signals corresponding to the 
test results. 

30 20. The diagnostic tester of claim 19, further comprising: 

control means (32) responsive to said output signal for producing said control signal to cause said automatic 
testing means to perform at least one further selected test on the system if said output signal indicates that insuf- 
ficient test data has been evaluated; and for producing a conclusion signal indicative of a diagnosis in accordance 
with said first candidate signal if said output signal indicates that sufficient test data has been evaluated. 

3S 

21. The diagnostic tester of claim 20, wherein said control means (32) produces said control signal to select said at 
least one further selected test in accordance with at least one prior evaluation of test signals. 

22. The diagnostic tester of claim 17 further comprising: 

40 

a display (12.22.36); and 

means (24) responsive to said output signal for controlling said display to indicate the need for additional test 
data and for controlling said input means to accept a further at least one test signal for evaluation by said 
evaluating means if said output signal indicates that Insufficient test data has been evaluated; and for control- 
45 ling said display responsive to said first candidate signal to indicate a diagnosis if said output signal indicates 

that sufficient test data has been evaluated. 

23. A diagnostic tester according to any of claims 17 to 22 and/or comprising apparatus according to any of claims 1 
to 1 6 the tester comprising an arrangement for testing a system according to a set of defined relationships between 

so a plurality of sources for providing information about the system and a plurality of conclusions about the nature or 

condition of the system, said tester comprising: 

first means (34) for successively selecting (551 ) an information source from among the plurality of Information 
sources for inl rrogation according to the relative amount of information provided by each information source 
ss until all of the information sourc s hav been select d or the information which would b provided thereby 

has been determined; 

second means (34) for successively further selecting an information source from among the plurality of infor- 
mation sources for interrogation according to which information sourc will increase th level of certainty that 
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a selected candidate conclusion constitutes a valid diagnosis of the system being tested. 

24. The diagn stic tester of claim 23 wherein said second s lecting means comprises: 

means for assigning a confidence value to the information provided by an information source; 
means responsive to said confidence values and to the information provided by interrogated information sourc- 
es for computing a level of certainty for each of the conclusions that it constitutes a valid diagnosis; 
means for selecting candidate conclusions according to the relative values of the computed levels of certainty 
for each conclusion. 

25. The diagnostic tester of claim 24 further comprising: 

first means for Identifying, if more than one candidate conclusion has been selected, which information sourc- 
es, if any. depend on one of the two candidate conclusions with the then highest levels of certainty and not the 
other, and. if more than one such information source is identified, identifying which of such information sources 
furnishes the most information about the support for the candidate conclusion on which it depends. 

26. The diagnostic tester of claim 25 further comprising: 

second means, responsive to said first identifying means determining that no information source satisfies 
the identification criteria of said first identifying means, and responsive to said selecting means having then selected 
only one candidate conclusion, for identifying the information source, if any. that provides the most support for the 
candidate conclusion having the then highest level of certainty, if more than one candidate conclusion has then 
been selected, or for the candidate conclusion, if only one candidate conclusbn has then been selected. 

27. The diagnostic tester of claim 26 further comprising: 

third means, responsive to said second identifying means determining that no information source satisfies 
the identification criteria of said second identifying means, for identifying the information source, if any, that supports 
the most conclusions other than the candidate conclusion having the then highest level of certainty 



Patentanspruche 

1. Automatische Testdatenauswertevorrichtung zur Verwendung in einem diagnostischen Tester, der auf Basis der 
Lokalisation oder isolation von versagten Systemelementen eines Systems mit einer definierten Anzahl funktlo- 
naier Elemente eine Diagnose abgibt, mit: 

einem Mittel zum Auswahlen des nachsten durchzufuhrenden Tests auf dem zu testenden System, 

einem Mittel zum Generieren der korrespondierenden Testdaten und zum Liefem derselben an das zu testende 
System, 

einem Mittel zum Wahrnehmen und Sammein vorbestimmter Systembetriebsparameter, 

einem Auswertungsmittel (54), das sensibel ist fur mindestens ein eingegangenes Testsignal. das mit Test- 
daten Sj korrespondiert, welches sich auf mindestens einen vorbestimmten Parameter des Systems bezieht, 
zum Produzleren unter Verwendung einer probabilistischen nachweisenden Begrundungsmethode (?), von 

ersten (S^p^ ) und zweiten (Sqp2) Kandidatensignalen, die jeweils mit ersten und zweiten moglichen Diagnosen 
des Zustands des Systems korrespondieren und jeweils den hochsten bzw. zweithochsten Bestimmtheitsgrad, 
gultig zu sein, haben und 

ersten (Sp^) und zweiten (Sp2) Bestimmtheitssignalen. die jeweils mit den Werten des hochsten bzw. zweit- 
hochsten Bestimmtheitsgrads korrespondieren. und 

gekennzeichnet durch Mitt I zum Testen der Hinlanglichkeit (56), sensibel fur die ersten und zweiten B stimmt- 
heltssignale, zum Produzieren ines Ausgangssignals (Sg), das anzeigt, ob hinlanglich vi le Testdaten ausgewer- 
tet worden sind. um in Diagnose abzugeben. wobei das Mittel zum Testen der Hinlanglichkeit in neuronales 
Netzwerk aufweist, mit einem Ausgangsknoten (130') und mindestens drei Eingangsknoten (IIV, 112', 114'), die 
jeweils die ersten und zweiten Bestimmtheitssignale und in vorher festgesetztes Biassignal als EIngang empfan- 
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gen, wobei die Eingangsknot n gewichtete Ausgang haben. die mit einem Modell aus Expertendaten k rrespon- 
dieren, wobei die Expertendaten Kombinationen von Bestimnr^theitssignalen mit mindestens einer Expertenmei- 
nung in bezug auf die Fahigk it, eine Diagnose auf Basis jeder Konnbinatlon von Bestlmmtheitssignalwerten ab- 
zugegen, korrelieren. 

Die Vorrichtung nach Anspruch 1 , wobei 

das Auswertungsmitte) auch ein Unbestimmtheitssignal (S^) produziert, das nnit einem MaB der Unbestimmt- 
heit korrespondiert. daB das ausgewertete mindestens eine Testsignal gultig ausgewertet werden kann. und 

das neuronale Netzwerk folgendes aufweist: 

einen Satz (110) erster (111), zweiter (112), dritter (113) und vierter (114) Eingangsknoten. von denen 
jeder erste. zweite und dritte Ausgange hat, die durch ein vorher testgesetztes. bestimmtes Gewicht ge- 
wichtet sind, wobei die ersten (111)undzweiten (112) Eingangsknoten jeweils die ersten (Sp^) und zweiten 
(Sp2) Beslimmtheitssignale als Eingang emptangen. der dritte Eingangsknoten (113) das Unbestimmt- 
heitssignal (S^) empfangt und der vierte Eingangsknoten (114) das vorher f estgesetztes Blassignal (BIAS) 
empfangt, 

einen Satz (120) erster (121). zweiter (122), dritter (123) und vierter (124) mittlerer Knoten, von denen 
jeder einen durch ein vorher testgesetztes bestimmtes Gewicht gewichteten Ausgang hat, wobei der erste 
mittlere Knoten (121) die Summe der ersten gewichteten Ausgange der Eingangsknoten als Eingang 
empfangt, der zweite mittlere Knoten (122) dieselben der zweiten gewichteten Ausgange der Eingangs- 
knoten als Eingang empfangt. der dritte mittlere Knoten (123) die Summe der dritten gewichteten Aus- 
gange der Eingangsknoten als Eingang empfangt und der vierte mittlere Knoten (1 24) das vorher f estge- 
setzte Blassignal (BIAS) empfangt, und 

einen Ausgangsknoten (130). der sensibel ist fur die Summe der gewk^hteten Ausgange der mittleren 
Knoten, zum Produzieren des Ausgangssignals (S3), wobei die Ausgangsgewichte der Eingangs- und 
mittleren Knoten mit einem Modell von Expertendaten korrespondieren, wobei die Expertendaten Kom- 
binationen von Bestimmtheits- und Unbestimmtheitssignalwerten mit mindestens einer Expertenmeinung 
bezuglich der Fahigkeit korrelieren, eine Diagnose auf Basis jeder Kombination von Bestimmtheits- und 
Unbestimmtheitssignalwerten abzugeben, wodurch das Mittel zum Testen der Hinlanglichkeit das Unbe- 
stimmtheitssignal ebenfalls auswertet, um das Ausgangssignal zu produzieren. 

Die Vorrichtung nach Anspruch 2, wobei 

das Auswertungsmittel auch ein Signal (Si^) produziert, das mit einem MaB des Bestimmtheitsgrads korre- 
spondiert. daB eine nicht mit den ersten oder zweiten Kandidatensignalen Oder mit dem Unbestimmtheitssignal 
korrespondierende Diagnose eine gultige Diagnose ist, und 

das Mittel zum Testen der Hinlanglichkeit auch das Signal (S|g) auswertet, um das Ausgangssignal zu produ- 
zieren. 

Die Vorrichtung nach Anspruch 2, wobei das Auswertungsmittel (54) folgendes aufweist: 

ein Mittel zum Speichern eines Modells von FluBbeziehungen vorher festgelegter Information Innerhalb des 
im Test befindlichen Systems, 

ein Mittel zum Assoziieren des Testsignals mit Satzen moglicher Ursachen in Obereinstimmung mit dem Modell 
und zum Ermittein der Schnittpunkte der Satze moglicher Ursachen fur Kombinatbnen von Testsignaten, wo- 
bei die Kandidatensignale von den Schnittpunkten hergeleitet werden, und 

ein Mittel zum B stimmen eines MaBes des Bestimmtheitsgrads fur jedes Kandidatensignat, daB es eine 
gultige Diagnose reprasentiert. 

Die Vorrichtung nach Anspruch 4. wobei das Auswertungsmittel (54) Mittel aufweist zum Bestlmmen des Grads, 
zu dem ein Konflikt zwischen mindestens dem ersten Kandidatensignal und mind st ns inem Testsignal existiert, 
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das den Bestimmtheitsgrad anzeigt, da3 In nicht antizipiertes Ergebnis (UR) erhatten worden ist, wobei das Un- 
bestimmtheitssignal einen mit dem UR-Bestimmtheltsgrad korrespondlerenden Wert hat. 

Die Vorrichtung nach Anspruch 1 . wobei die Werte der hochsten und zweithochsten Bestimmtheitsgrad WSahr* 
scheinlichkeitswerte sind. 

Die Vorrichtung nach Anspruch 2, wobei das Unbestimmtheitsmaf) einen Wahrscheiniichkeitswert hat. 

Die Vorrichtung nach Anspruch 5, wobei das UR-Bestimmungsmittel folgendes aufweist: 

ein Mittel zum Definieren einer Vielzahl von SchtuBfotgerungen, die mit moglichen Diagnosen des Systems 
korrespondieren, 

ein Mittel zum Zuweisen eines Vertrauenswerts zu jedem fortfolgend eingegangenen Testsignal. 

ein Mittel zum Berechnen der Zustimmung und Ablehnung fur jede SchluBfolgerung baslerend auf dem Ver- 
trauenswert des gegenwartig eingegangenen Testsignals. 

ein Mittel zum Bestimmen, ob das gegenwartig eingegangene Testsignal einem dann ausgewahlten Satz der 
mit Kandidatensignalen korrespondlerenden SchluBfolgerungen widerspricht, 

ein Mittel zum Bestimmen eines Ma3es internen Widerspruchs basiert auf akkumulierten totalen Zustlm- 
mungswerten und den gegenwartigen Testsignalstutzwerten fur atle SchluBfolgerungen, 

ein Mittel zum Berechnen des gegenwartigen Werts des Unbestimmtheitssignals basiert auf einer Anzahl 
ausgewerteter Testsignale, wenn das gegenwartige Testsignal nIcht einen Wlderspruch bildet. sowie basiert 
auf der Anzahl ausgewerteter Testsignale und der Anzahl WIderspruche. wenn das gegenwartige Testsignal 
einen Wlderspruch bildet, 

ein Mittel zum Berechnen eines neuen totalen Zustimmungswerts fur jede SchluBfolgerung basiert auf dem 
alten totalen Zustlmmungswert. dem gegenwartigen Testslgnatstutzwert und Vertrauenswert, einem alten Un- 
bestlmmtheitswert und dem MaB internen Widerspruchs, 

ein Mittel zum Berechnen eines neuen Unbestimmtheitswerts basiert auf dem alten Unbestlmmtheitswert, 
dem gegenwartigen Testslgnalvertrauenswert und dem MaB Internen Widerspruchs, 

ein Mittel zum Normalisieren des totalen Zustimungswerts fur jede SchluBfolgerung basiert auf einer alten 
totalen Ablehnung und dem gegenwartigen Testsignalablehnungswert, 

ein Mittel zum Berechnen der gegenwartigen Plauslbilitat fur jede SchluBfolgerung basiert auf der gegenwar- 
tigen totalen Ablehnung und der Anzahl ausgewerteter Testsignale und 

ein Mittel zum Auswahlen des gegenwartigen Satzes Kandidatensignale basiert auf der gegenwartigen totalen 
Zustimmung (auch vorher) und der Plauslbilitat fur jede SchluBfolgerung. 

Vorrichtung nach Anspruch 2, wobei die Ausgangsgewlchte der Eingangs- (111 bis 114) und mittleren (121 bis 
1 24) Knoten Werte haben, die durch Training eines ruckwarts verzweigten neuronalen Netzwerks ermittelt worden 
sind. das die gleiche Archltektur hat wie das neuronale Netzwerk. das empirlsche Bestimmthelts- und Unbestimm- 
heitssignal-Tralningsdaten. die belm Testen realer Systeme ermittelt wurden, sowie SchluBfolgerungen verwendet, 
die von mindestens einem Experten bezuglich der HInlanglichkeit der von den Komblnationen reprasentierten 
Information hergeleitet sind, um eine Diagnose fur ein spezifisches zu testendes System abzugeben. 

Die Vorrichtung nach Anspruch 9. wobei die Ausgangsgewlchte der Eingangs- (111 bis 114) und mittleren (121 
bis 124) Knoten generische W rte haben, die durch Training eines ruckwarts verzweigten neuronalen Netzwerks 
bestimmt worden sind, das dieselbe Archltektur hat wie das neuronale Netzwerk, das normalisiert . zufallig ge- 
neri rte Bestrmmtheits- und Unbestimmheitssignal-Trainingsdaten verwendet sow! SchluBfolgerungen. die von 
mindestens einem Experten bezuglich der HInlanglichkeit der von den Komblnationen reprasentierten Information 
hergeleitet worden sind, um eine Diagnose fur Systeme im allgemeinen abzugeben. 
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11. Die V rrichtung nach Anspruch 5. wobei die Werte der Bestimmtheits- und Unbestlmmtheitssignal mit W!ahr- 
scheinlichkeitswerten korrespondieren. 

12. Die Vorrichtung nach Anspruch 11 , wobei di Train in gsdaten rste, zweite unddrrtt unzweideutige Kombinationen 
s aulweisen, in denen die Datenhintanglichkeitsfestlegung offensichtlich ist, wob i die erste unzweideutige Kombi- 

natbn das rste Bestinnmth itssignal aufw ist, das einen Wert hat, der mit 100% Wahrscheinlichkeit korrespon- 
diert. sowie das zweite Bestimnntheilssignal und das Unbestimnntheitssignal aufweist, die Nullwerte haben. und 
wobei die zweite unzweideutige Komblnation das erste Bestimnntheitssignai und das Unbestimnntheitssignal auf* 
weist. von denen jedes einen Wert hat. der mit 100% Wahrscheinlichkeit korrespondiert, sowie das zweite Be- 
10 stimmtheitssignal autweist. das einen Nuliwert hat, und wobei die dhtte unzweideutige Kombination beide Be- 

sttmmtheitssignale aufweist. die mit einheitiichen WSahrscheinlichkeiten korrespondieren, sowie das Unbestimmt- 
heitssignal aufweist, das einen Nuliwert hat. 

13. Die Vorrichtung nach Anspruch 12, wobei die Trainingsdaten zwei Datensatze aufweist, von denen jeder eine 
IS vorbestimmte Anzahl zweideutiger Kombinationen hat, wobei: 

•. 

beide Datensatze Bestimmtheitssignalwerte innerhalb eines geschiossenen Inten^alls [0,1] haben, die ent- 
sprechend einer logarithmischen Normalverteilung zufallig generiert sind, 

20 ein erster der Datensatze Unbestimmtheitssignalwerte hat, die entsprechend einer togarithmischen Nornnal- 

verteilung zufallig generiert sind, 

ein zweiter der Datensatze Unbestimmtheitssignalwerte hat. die zu Null gesetzt sind. 

2S die so generierten Werte durch Normalisierung der Bestimmtheitssignalwerte in jeder zweideutigen Kombi- 

nation eingestellt sind als Summe (1 • Sy) zu ergeben, wobei alle normaiisierten Bestimmtheitssignalwerte 
und die Unbestimmtheitssignalwerte. welche klelner oder glelch einem vorbestimmten Wert sind, der auf der 
Anzahl der Elemente im System basiert, zu Null gesetzt sind, und dann die resultierenden Bestlmmtheits- und 
Unbestimmtheitssignalwerte in jeder der zweideutigen Kombinationen normalisiert sind, um als Summe den 

30 Wert eins zu ergeben, und 

der groBere der resultierenden normaiisierten Bestimmtheitssignalwerte als der erste Bestimmtheitssignalwert 
in jeder der zweideutigen Kombinationen bezeichnet wird, in der die Bestimmtheitssignalwerte ungleich sind. 

35 14. Die Vorrichtung nach Anspruch 11. wobei 

die Summe der gewichteten, jedem der mittleren Knoten eingegebenen Ausgange und der Ausgangsmodus 
im jeweiligen Knoten mit einer sigmoidalen Aktivierungsfunktion verarbeitet werden. 

"^0 die Gewichte der ersten. zweiten und dritten Ausgange des ersten Eingangsknotens jeweils (-9,640), 5,049 

und (-1 ,624) betragen, 

die Gewichte des ersten, zweiten und dritten Ausgangs des zweiten Eingangsknotens jeweils 1 5,641 , (-8.491 ) 
und 2,436 betragen, 

4S 

die Gewichte des ersten. zweiten und dritten Ausgangs des dritten Eingangsknotens jeweils (-6,680). 3,627 
und (-1,237) betragen, 

die Gewichte des ersten, zweiten und dritten Ausgangs des vierten Eingangsknotens jeweils (-0,422). 0.442 
so und (-0.427) betragen. 

das Gewicht des Ausgangs des ersten mittleren Knotens (-19,214) betragt, 

das Gewicht des Ausgangs des zweiten mittleren Knot ns 11,457 b tragt, 

55 

das Gewicht des Ausgangs des dritten mittleren Knotens (-2,483) b tragt und 
das Gewicht des Ausgangs des vierten mittleren Knotens 4,081 betragt. 
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15. Die Vorrichtung nach Anspruch 14, wobei die Aktivierungsfunktion logistisch ist. 

16. Die Vorrichtung nach Anspruch 15. wobei die Aktivierungsfunktion 1/(1-e.x) lautet, mit x als ungewichtetem Aus- 
gang eines geg ben n Knotens. 

5 

17. Ein diagnostischer Tester zum Abgeben ein r Diagnose auf Basis der Lokalisation und Is lation versagter Sy- * 
stennelemente eines Systenns mit einer definierten Anzahl f unktionaler Elennente, wobei der Tester die Vorrichtung 
nach jedem beliebigen der vorhergehenden Anspruche aufweist sowie Eingangsmittel (13. 26. 34, 42 52) zum 
Llefem von Testsignalen an das Auswertungsmittel. 

10 

18. Der diagnostische Tester nach Anspruch 17. wobei das Eingangsmittel operatoraktivierte Eingangsmittel (10. 26) 
zum Eingeben von Testdaten aufweist, die sich auf mindestens einen vorbestimmten Aspekt des Systems bezie- 
hen, sowie zum Generieren von Testsignalen. die mit den Testdaten korrespondieren. 

^5 19. Der diagnostische Tester nach Anspruch 1 7, wobei das Eingangsmittel ein automatisches Testmittel (34) aufweist, 
das zum Durchfuhren ausgewahlter Tests auf dem System und zum Generieren von Testsignalen, die mit den 
Vestergebnissen korrespondieren, sensibel fur ein Kontrollsignal ist. 

20. Der diagnostische Tester nach Anspruch 19. zusatzlich mit: 

so einem Kontroiimittel (32), das zum Produzieren des Kontrollsignals sensibel fur das Ausgangssignal ist, um das 

Kontrollsignal zu produzieren, um zu erwirken, dalB das automatische Testmittel mindestens einen weiteren aus- 
gewahlten Test aus dem System durchf Qhrt, wenn das Ausgangssignal anzeigt. daf3 nicht hinlanglich viele Test- 
daten ausgewertet worden sind, sowie um ein SchluOfolgerungssignal zu produzieren, das in Ubereinstimmung 
mit dem ersten Kandidatenslgnal eine Diagnose anzeigt. wenn das Ausgangssignal anzeigt, daB hinlanglich viele 

25 Testdaten aufgewertet worden sind. 

21. Der diagnostische Tester nach Anspruch 20. wobei das Kontroiimittel (32) das Kontrollsignal zum Auswahlen des 
mindestens einen weiteren ausgewahlten Tests In Ubereinstimmung mit mindestens einer vorhergehenden Aus- 
wertung der Testsignale produziert. 

30 

22. Der diagnostische Tester nach Anspruch 17 zusatzlich mit: 

einer Anzeige (12. 22, 36) und 

35 einem MIttel (24). das sensibel fur das Ausgangssignal ist, um die Anzeige zu steuern, die Notwendigkeit 

zusatzlicher Testdaten anzuzeigen, und umdas Eingangsmittel zu steuern, mindestens ein weiteres Testsignal 
zur Auswertung durch das Auswertungsmittel zu akzeptieren, wenn das Ausgangssignal anzeigt, daB nicht 
hinlanglich viele Testdaten ausgewertet worden sind. und um die fur das erste Kandidatenslgnal sensible 
Anzeige zu steuern, eine Diagnose anzuzeigen, wenn das Ausgangssignal anzeigt. daB hinlanglich viele Test- 

40 daten ausgewertet worden sind. 

23. Ein diagnostischer Tester nach einem der Anspruche 17 bis 22 und/oder mit einer Vorrichtung nach einem der 
Anspruche 1 bis 16, wobei der Tester eine Anordnung zum Testen eines Systems entsprechend einem Satz defi- 
nierter Beziehungen zwischen einer Anzahl von Quellen zum Versorgen mit Informationen uber das System und 

45 einer Anzahl von SchluBfolgerungen uber die Natur oder den Zustand des Systems aufweist. sowie: 

ein erstes Mittel (34) zum fortfolgenden Auswahlen (551) einer Informationsquelle unter der Anzahl der Infor- 
mationsquellen zur Abfrage entsprechend der relativen Menge an Informationen. die von jeder Informations- 
quelle geliefert ist. bis alle Informationsquellen ausgewahit worden sind Oder die Information, die dadurch 
so geliefert wQrde, bestimmt worden ist, 

ein zweites Mittel (34) zum fortfolgenden weiteren Auswahlen einer Informationsquelle aus einer Anzahl von 
Informationsquellen zum Abfragen entsprechend dem Kriterium, welche Informationsquelle den Bestimmt- 
heitsgrad ansteigen laBt, daB eine ausgewahlte SchluBfolgerung zu einem Kandidaten eine gultige Diagnose 
ss des getesteten Systems bildet. 

24. Der diagnostische Tester nach Anspruch 23, wobei das zweite Auswahlmittel folgendes aufweist: 



29 



EP0 474 944B1 



ein Mittel zum Zuweisen eines Vertrauenswerts zur Information, die von einer Informationsquelle geliefert 1st. 

ein Mittel, das senslbel ist fur die Vertrauenswerte und fur die Infornnatlon. die von abgefragten Informations- 
quellen geliefert ist, urn einen Bestimmtheitsgrad fur j de der SchluBfolgerungen zu berechnen, da3 er eine 
5 gultige Diagnose bildet. 

ein Mittel zum Auswahlen von SchluBfolgerungen zu einem Kandidaten entsprechend den relativen Werten 
der berechneten Bestimmtheitsgrade fur jede SchluBfolgerung. 

10 25. Der diagnostische Tester nach Anspruch 24 zusatzlich mit: 

einem ersten Mittel zum Identifizieren. ob mehr als eine SchluBfolgerung zu einem Kandidaten ausgewahit worden 
ist, welche Informationsquellen, wenn uberhaupt eine, von einer der beiden SchluBfolgerungen zum Kandidaten 
nr>it den dann hochsten Bestimmtheitsgraden abhangen und nicht von der anderen, und, wenn mehr als eine solche 
Informationsquelle identifiziert ist. zum Identifizieren, welche der Infomriationsquellen die meiste Information ge- 

IS wahrt uber die Zustimmung der SchluBfolgerung zum Kandidaten. von der sie abhangt. 

26. Der diagnostische Tester nach Anspruch 25 zusatzlich mit: 

einem zweiten Mittel, das sensibel ist fur das erste Identifizierungsmittel. welches festlegt, daB keine Informati- 
onsquelle dem Identifikationskrrterium des ersten Identifizierungsmittels genugt. und sensibel ist fur das Auswahl- 
20 mittel, das dann nur eine SchluBfolgerung zum Kandidaten ausgewahtt hat, um die Informationsquelle. wenn es 

eine gibt, zu identifizieren, die die meiste Zustimmung fOr die SchluBfolgerung zum Kandidaten mit dem dann 
hochsten Bestimmtheitsgrad liefert, wenn dann mehr als eine SchluBfolgerung zum Kandidaten ausgewahit wor- 
den ist, Oder fur die eine SchluBfolgerung zum Kandidaten. wenn dann nur eine SchluBfolgerung zum Kandidaten 
ausgewahit worden ist. 

2S 

27. Der diagnostische Tester nach Anspruch 26 zusatzlich mit: 

einem dritten Mittel. das sensibel ist fur das zweite Identifizierungsmittel, welches bestimmt, daB keine Informati- 
onsquelle dem Identifikationskriterium des zweiten Identifizierungsmittels zum Identifizieren der Informationsquel- 
le. wenn es eine gibt. genugt, die die meisten von der SchluBfolgerung zum Kandidaten verschiedenen 
30 SchluBfolgerungen unterstutzt, die dann den hochsten Bestimmtheitsgrad haben. 



R vendlcations 

1. Appareil automatique d'^valuation de donn^es de test pour utilisation dans un appareil de test de diagnostic qui 
dtabirt un diagnostic sur la base d'une localisation ou d'une isolation d'dl^ments de systdme ddfaillants d*un sys- 
tdme ayant un nombre defini d'dldments fonctionnels. ledit appareil comprenant : 

des moyens pour s^lectionner le test suivant k effectuer sur le syst^me ^ tester ; 

des rhoyens pour g^nerer les donnees de test correspondantes et pour les fournir au syst^me k tester ; 

des moyens pour d6tecter et recueillir des param6tres pr6d6termin6s de fonctionnement du syst6me ; 

des moyens d'6valuation (54), sensibles k au moins un signal de test entre correspondant aux donn6es de 

test Sj concernant au moins un param^tre pr6d6termin6 du systdme. pour produire en utilisant une m^thode 

de raisonnement dvidentiel probabifiste : 

des premier (S^p^) et deuxi^me (Scp2) signaux candidats correspondant respectivement k des premier 
et deuxi^me diagnostics possibles de I'^tat du syst^me ayant respectivement les premier et deuxidme 
plus hauts niveaux de certitude d'§tre valables, et 

des premier (Sp^) et deuxidme (Sp2) signaux de certitude correspondant respectivement k des valeurs 
desdits premier et deuxl^me plus hauts niveaux de certitude ; et 

caract^ris^ par des moyens de suffisance des tests (56), sensibles auxdits premier et deuxr^me signaux de 
certitude, pour produire un signal de sortie (Ss) indiquant si assez de donnees de test ont 6t6 evalu6es pour 6tablir 
un diagnostic, I sdits moyens de suffisance des tests comprenant un r^seau neuronal ayant un noeud de sortie 
(130*) et au moins trois noeuds d'entr^e (111 \ 112'. 114') recevant respectivement lesdits premier t deuxidm 
signaux de certitude et un signal de polarisation pr6determin6 en tant qu'entr6 , lesdits noeuds d'entr6e ayant 
des sonies pond^r^es, les ponddrations de sortie corr spondant k un module d donndes d'expert mettant en 
correlation des combinaisons de signaux de c rtitud avec au moins une opinion d' xpert concernant la capacity 
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d'dtablir un diagnostic sur la base de chaque combinaison de val urs de signaux d c rtitude. 

Appareil selon la revendication 1 , dans lequ I : 

lesdits moyens d'evaluation produisent aussi un signal d'incertitude Sy correspondant d une mesure de Tin- 
c rtitude qu*au moins un signal de test dvalud peut dtre dvalud de fa^on valable ; et 
ledit r^seau neuronal con^prend : 

un ensemble (110) de premier (111), deuxi^me (112), troisieme (113) et quatri§me (114) noeuds d'entr6e, 
chacun ayant des premiere, deuxt^me et troisieme sorties ponddr^es par une ponddration fixe pr6d6ter- 
min^e. lesdits premier (111) et deuxi^me (112) noeuds d*entr6e recevant respectivement lesdits premier 
(Sp^) et deuxieme (Spg) signaux de certitude en tant qu'entree, ledit troisieme (113) noeud d*entr6e rece- 
vant ledit signal d'incertitude (S^), et ledit quatridme noeud d'entr6e (114) recevant ledit signal de polari- 
sation prdddtermind (BIAS) ; 

un ensemble (120) de premier (121). deuxieme (122), troisieme (123) et quatri^me (124) noeuds inter- 
mediaires. chacun ayant une sortie ponder^e par une pond^ration fixe pred6termin6e. et ledit premier 
noeud intermddiaire (121) recevant la somme des premieres sorties ponddrdes desdits noeuds d'entrde 
en tant qu'entree, ledit deuxieme noeud interm6diaire (122) recevant celle des deuxi^mes sorties pond6- 
r6es desdits noeuds d'entr^e en tant qu'entree, ledit troisieme noeud intermediaire (123) recevant la som- 
me des troisi^mes sorties pond^r^es desdits noeuds d'entr^e en tant qu'entree, et ledit quatri^me noeud 
intermediaire (124) recevant ledit signal de polarisation (BIAS) pr^d^termind ; et 
un noeud de sortie (1 30), sensible k la somme desdites sorties pond^rSes desdits noeuds interm^diaires, 
pour produire ledit signal de sortie (S3), les pond^rations de sortie desdits noeuds d'entr^e et intermd- 
diaires correspondant a un modele de donnees d'expert mettant en correlation des combinaisons de va- 
leurs de signaux de certitude et d'incertitude avec au molns une opinion d'expert concernant la capacity 
d'6tablir un diagnostic sur la base de chaque combinaison de valeurs de signaux de certitude et d'incer- 
titude, lesdits moyens de suffisance des tests evaluant ainsi aussi ledit signal d'incertitude pour produire 
ledit signal de sortie. 

Appareil selon la revendication 2. dans lequel : 

lesdits moyens d'dvaluation produisent aussi un signal S|sj correspondant ^ une mesure du niveau de certitude 
qu'un diagnostic ne correspondant pas auxdits premier ou deuxieme signaux candidats ni audit signal d'in- 
certitude est un diagnostic valable ; et 

lesdits moyens de suffisance des tests evaluent aussi ledit signal S^ pour produire ledit signal de sortie. 

Appareil selon la revendication 2. dans lequel lesdits moyens d'evaluation (54) comprennent : 

des moyens pour stocker un module de relations predetermindes de flux d'information dans le syst^me soumis 
au test ; 

des moyens pour associer lesdits signaux de test k des ensembles de causes possibles conformement audit 
module, et pour determiner les intersections desdits ensembles de causes possibles pour des combinaisons 
de signaux de test, lesdits signaux candidats etant derives ^ partir desdites intersections ; et 
des moyens pour determiner pour chaque signal candidat une mesure du niveau de certitude qu'il represente 
un diagnostic valable. 

Appareil selon la revendication 4, dans lequel lesdits moyens d'evaluation (54) comprennent des moyens pour 
determiner jusqu'^ quel degre un conflit existe entre ledit au moins un premier signal candidat et ledit au moins 
un signal de test indiquant le niveau de certitude qu'un resultat non anticipe (UR) a ete obtenu, ledit signal d'in- 
certitude ayant une valeur correspondant audit niveau UR de certitude. 

Appareil selon la revendication 1 , dans lequel les valeurs desdits premier et deuxieme plus hauts niveaux de 
certitude sont des valeurs de probabilite. 

Appareil selon la r v ndication 2. dans lequel ladite mesure d'incertitude a une valeur de probabilite. 
Appareil s ion la revendication 5, dans lequel lesdits moyens de determination d'UR comprennent : 
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des moyens pour dSfinir une pluralite de conclusions correspondant k des diagnostics possibles du systdme ; 
des moyens pour affecter une valeur de confiance h chaque signal de test successlvement entrd ; 
des moyens pour calculer la confirmation et le rej t pour chaque conclusion basde sur tadite valeur de con- 
fiance du signal de test actu llement entrd ; 
5 des moy ns pour determiner si le signal de test actuellement entr6 est en conf lit avec un ensemble. s61ectionn6 

k ce moment-!^, desdites conclusions correspondant aux signaux candidats ; 

des moyens pour obtenir une mesure de conflh interne bas4e sur des valeurs cumulees de confirmation totale 
et les valeurs de confirmation du signal de test actuel pour toutes lesdites conclusions ; 
des moyens pour calculer la pr^sente valeur dudit signal d'incertitude bas^e sur le nombre de signaux de test 
10 dvaluds, si le present signal de test n'est pas un conflit ; et. bas6e sur le nombre de signaux de test dvaluds 

et sur le nombre de conflits, si le present signal de test est un conflit ; 

des moyens pour calculer une nouvelle valeur de confirmation totale pour chaque conclusion basde sur Tan- 
cienne valeur de confirmation totale. la valeur de confirmation et la valeur de confiance du present signal de 
test, une anctenne valeur d'incertttude, et ladite mesure de conflit Interne ; 
IS des moyens pour calculer une nouvelle valeur d'incertitude basSe sur I'ancienne valeur d'incertitude. ta valeur 

de confiance du present signal de test et ladite mesure de conflit inteme ; 

des moyens pour nomnaliser la valeur de confirmation totale pour chaque conclusion en fonction de I'ancienne 

valeur de confirmation totale, et de la valeur du present signal d'incertitude ; 

des moyens pour calculer un nouveau rejet total pour chaque conclusion en fonction d'un ancien rejet total et 
20 de la valeur de rejet du present signal de test ; 

des moyens pour calculer la prdsente vraisemblance pour chaque conclusion bas^e sur le present rejet total 
et le nombre de signaux de test dvalu^s ; 

des moyens pour s^lectionner I'ensemble actuel de signaux candidats en fonction de ta prdsente confirmation 
totale et de la vraisemblance pour chaque conclusion. 

9. Appareil selon la revendication 2, dans lequel lesdites pond6rations de sortie desdits noeuds d'entr6e (111-114) 
et interm6diaires (121-124) ont des valeurs obtenues par apprentissage d'un reseau neuronal ^ propagation en 
retour, ayant la meme architecture que ledit reseau neuronal, en utilisant des donndes d'apprentissage de signaux 
de certitude et d'incertitude empiriques obtenues par des tests sur des systdmes r6els, et des conclusions d^rivdes 

30 d'au moins un expert concemant la suffisance des informations representees par lesdites combinatsons pour 

etabiir un diagnostic pour un systeme specifique k tester 

10. Appareil selon la revendication 9. dans lequel lesdites ponddrations de sortie desdits noeuds d'entree (111-114) 
et intermediaires (121-124) ont des valeurs g6n6riques obtenues par apprentissage d'un r6seau neuronal k pro- 

3S pagation en retour, ayant la meme architecture que ledit reseau neuronal, en utilisant des donnees d'apprentissage 

normalisees pour le signal de certitude et d'incertitude generees de maniere aieatolre, et des conclusions derlvees 
d'au moins un expert concernant la suffisance des informations representees par lesdites combinaisons pour 
etablir un diagnostic pour des syst^mes en general. 

40 11. Appareil selon la revendication 5, dans lequel les valeurs desdits signaux de certitude et d'incertitude correspon- 
dent d des valeurs de probabilite. 

12. Appareil selon la revendication 11 . dans lequel lesdites donnees d'apprentissage incluent des premiere, deuxieme 
et troisieme combinaisons sans ambiguTte dans lesquelles la determination de la suffisance des donnees est 

45 evidente, ladite premiere combinaison sans ambiguite comprenant ledit premier signal de certitude ayant une 

valeur correspondant ^ une probabilite de 100 %, et ledit deuxieme signal de certitude et ledit signal d'incertitude 
ayant des valeurs de zero, ladite deuxieme combinaison sans ambiguite comprenant ledit premier signal de cer- 
titude et ledit signal d'incertitude, chacun ayant une valeur correspondant k une probabilite de 100 %, et ledit 
deuxieme signal de certitude ayant une valeur de zero, et ladite troisieme combinaison sans ambiguite connprenant 

so les deux signaux de certitude ayant des valeurs correspondant ^ des probabilites unifonmes et ledit signal d'incer- 

titude ayant une valeur de zero. 

13. Appareil selon la revendication 12, dans lequel lesdites donnees d'apprentissage incluent deux ensembles de 
donnees, chacun ayant un nombre predetermine de combinaisons ambigues ou : 

55 

les deux ens mbles de donnees ont des valeurs de signaux de certitud dans un intervene ferme [0,1] gene- 
rees au hasard selon une distribution log-normale ; 

un premier desdits ensembles de donnees a des valeurs de signaux d'incertitude generees au hasard selon 
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une distribution log-normale ; 

un deuxiem desdits ensembles de donnees a des vaieurs de signaux d'incertitude Stabiles ^ z6ro ; 

les vaieurs ainsi gdndrdes sont ajustdes en normalisant les vaieurs d signaux de certitude dans chaque 

combinaison ambigue pour totaliser (l-Sy), toutes les vaieurs normatis^es de signaux de certitude et les 

5 vaieurs de signaux d'incertitude inferieures ou egales ^ une valeur pr^^t rminde bas6e sur le nombre d'616' 

ments dans le systdm sont Stabiles d zdro ; et alors les vaieurs rdsuttantes des signaux de certitude et d'in- 
certitude dans chacune des combinaisons annbigues sont normalis^es pour totaliser la valeur un ; et 
la plus grande des vaieurs normalisees r^sultantes des signaux de certitude est d6sign6e comme la premiere 
valeur de signaux de certitude dans chacune desdites connbinaisons ambigues dans lesquelles les vaieurs 

10 de signaux de certitude sont in^gales. 

14. Appareil selon la revendication 11 . dans lequel : 

la somme des sorties ponddrSes entries k chacun des noeuds intermddiaires et au noeud de sortie est traitde 
IS dans le noeud respectif par une fonction deactivation sigmoide ; 

les pond^rations des premiere, deuxi^me et trolsieme sorties du premier noeud d'entr6e sont (-9,640), 5,049. 
et (-1,624), respectivement ; 

les pond^rations des premiere, deuxi^me et trolsieme sorties du deuxi^me noeud d'entr^e sont 15.641. (- 
8,491 ), et 2.436, respectivement ; 
20 les pond^rations des premiere, deuxieme et troisi^me sorties du troisi^me noeud d'entrSe sont (-6.680). 3.627, 

et (-1,237). respectivement ; 

les pond^rations des premiere, deuxieme et troisi^me sorties du quatridme noeud d*entr6e sont (-0.422), 
0,442, et (-0,427), respectivement ; 

la pond6ration de la sortie du premier noeud intermddiaire est (-19.214) ; 
2S la ponddration de la sortie du deuxidme noeud intemnddiaire est 11 .457 ; 

la pond6ration de la sortie du troisi^me noeud interm^diaire est (-2,483) ; et 
la pond§ration de la sortie du quatri^me noeud interm6diaire est 4,081. 

15. Appareil selon la revendication 14. dans lequel ladite fonction deactivation est logistique. 

30 

16. Appareil seton ta revendication 15, dans lequel ladite fonction d'activation est 1/(1-i-e'^). ou x est la sortie non 
pond6r6e d'un noeud donn6. 

17. Appareil de test de diagnostic pour ^tablir un diagnostic sur la base d'une localisation et isolation de composants 
35 de syst6me d6faillants d'un systeme ayant un nombre d6fini d'el^ments tonctionnels, I'appareil de test comprenant 

I'appareil selon I'une quelconque des revendications pr^c^dentes, I'appareil de test comprenant en outre des 
moyens d'entree (13. 26, 34. 42, 52) pour fournir des signaux de test auxdits moyens d'^valuation. 

18. Appareil de test de diagnostic selon la revendication 17, dans lequel lesdits moyens d'entree comprennent des 
"fo moyens d'entree (1 3, 26) actionnes par I'op^rateur pour entrer des donnees de test concernant au moins un aspect 

pr^d^termin^ du systeme, et pour gen^rer des signaux de test correspondent aux donnees de test. 

19. Appareil de test de diagnostic selon la revendication 17, dans lequel lesdits moyens d'entree comprennent des 
moyens de test automatiques (34) sensibles k un signal de commande pour effectuer des tests s6lectionn§s sur 

4S le systeme, et pour g^n^rer des signaux de test correspondant aux resultats de test. 

20. Appareil de test de diagnostic selon la revendication 19, comprenant en outre : 

des moyens de commande (32) sensibles audit signal de sortie pour produire ledit signal de commande pour 
amener lesdits moyens de test automatiques d effectuer au moins un autre test sdlectionnd sur le systeme si ledit 
so signal de sortie indique que des donnees de test insuffisantes ont 4t6 evalu^es ; et pour produire un signal de 

conclusion indiquant un diagnostic conform^ment audit premier signal candidal si ledit signal de sortie indique 
que des donnees de test suffisantes ont dvaludes. 

21 . Appar il de test de diagnostic selon la revendication 20, dans lequel lesdits moyens de commande (32) produisent 
ss ledit signal de commande pour s6lectionner ledit au moins un autre test s6lectionn6 conformdment k au moins 

un Evaluation ant^rieure des signaux de test. 

22. Appareil de test de diagnostic selon la revendication 17, comprenant n outr : 



33 



EP 0 474 944 B1 



un affichage (12. 22. 36) ; et 

des moyens (24) sensibles audit signal de sortie pour commander (edit affichage pour indiquer le besoin pour 
des donn^es de t st suppl^mentaires et pour commander lesdits moy ns d'entr^e de fa^on k accept r au 
moins un autre signal de t st ^ evaluer par lesdits moyens d'^valuation si ledit signal de sortie indique que 
5 des donnees de test insuffisantes ont 6te evaluees ; et pour commander ledit affichage sensible audit premier 

signal candidat de fagon k indiquer un diagnostic si ledit signal de sortie indique que des donnees de test 
suffisantes ont 6t6 6valu6es. 

23. Appareil de test de diagnostic selon Tune quelconque des revendications 17 d 22, et/ou comprenant un appareil 
10 selon Tune quelconque des revendications 1^16. I'appareil de test comprenant un agencement pour tester un 

syst^me selon un ensemble de relations d^finies entre une plurality de sources pour foumir des informations 
concernant te syst^me et une plurality de conclusions concernant la nature ou I'^tat du syst^me. ledit appareil de 
test comprenant : 

75 des premiers moyens (34) pour s6lectionner successivement (551 ) une source d'informations parmi la plurality 

de sources d'informations pour i'interroger selon la quantite d'information relative fournie par chaque source 
d'informations jusqu'd ce que toutes les sources d'informations aient 6t6 sdlectionndes ou que les Informations 
qui seraient fournies par celles-ci aient 6td d^termindes ; 

des deuxi^mes moyens (34) pour s^lectionner encore une fols successivement une source d'informations 
20 parmi la plurality de sources d'informations pour I'interroger selon la source d'informations qui augmentera le 

niveau de certitude qu'une conclusion candidate choisie constitue un diagnostic valable du syst^me testd. 

24. Appareil de test de diagnostic selon la revendication 23, dans lequel lesdits deuxl^mes moyens de selection 
comprennent : 

25 

des moyens pour affecter une valeur de confiance aux informations fournies par une source d'informations ; 
des moyens sensibles auxdites valeurs de confiance et aux informations fournies par des sources d'informa- 
tions interrogees pourcalculer un niveau de certitude pour chacune des conclusions qu'elle constitue un dia- 
gnostic valable ; 

30 des moyens pour s^lectionner des conclusions candidates selon les valeurs relatives des niveaux de certitude 

calculus pour chaque conclusion. 

25. Appareil de test de diagnostic selon la revendication 24 comprenant en outre : 

des premiers moyens pour identifier, si plus d'une conclusion candidate a 6te selectionn6e, quelles sources 
3S d'informations, le cas 6ch§ant, dependent d'une des deux conclusions candidates avec alors les plus hauts niveaux 

de certitude et pas de I'autre, et, si plus d'une telle source d'informations est identifi^e, pour identifier laquelle de 
ces sources d'informations fournit le plus d'information concernant la confirmation pour la conciusbn candidate 
dont eile depend. 

40 26. Appareil de test de diagnostic selon la revendication 25 comprenant en outre : 

des deuxi^mes moyens, sensibles auxdits premiers moyens d'identification determinant qu'aucune source 
d'informations ne satisfait aux criteres d'identification desdits premiers moyens d'identification. et sensibles auxdits 
moyens de s6lection ayant alors s6lectionn6 seulement une conclusion candidate, pour identifier la source d'in- 
formatbns, le cas ^chdant, qui fournit le plus de confirnnation pour la conclusion candidate ayant alors le plus haut 

45 niveau de certitude, si plus d'une conclusion candidate a alors ete s^lectionn^e, ou pour la conclusion candidate, 

si seulement une conclusion candidate a alors 6t6 s6lectionn6e. 

27. Appareil de test de diagnostic selon la revendication 26 comprenant en outre ; 

des trotsifemes moyens. sensibles auxdits deuxi^mes moyens d'identification determinant qu*aucune source 
so d'informations ne satisfait aux criteres d'identification desdits deuxiemes moyens d'identification, pour identifier la 

source d'informations, le cas 6ch6ant, qui confirme le plus grand nombre de conclusions autres que la conclusion 
candidate ayant alors le plus haut niveau de certitude. 
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